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PREFACE 


This  report  describes  the  investigation  of  potential  training- technology 
modernization  measures  within  Specialized  Undergraduate  Pilot  Training  (SUPT).  The 
research  was  conducted  by  the  Aircrew  Training  Research  Division,  Human  Resources 
Directorate,  of  Armstrong  Laboratory  (AL/HRA)  at  the  request  of  the  Air  Education  and 
Training  Command  (AETC/XOR).  Technologies  recommended  as  training  solutions  are 
based  on  the  opinions  of  modeling  and  simulation  experts  familiar  with  the  problems  and 
challenges  of  aircrew  training  in  the  Air  Force.  Recommended  implementation  methods 
are  based  on  theoretically  and  empirically  supported  training  principles  and  concepts. 

The  intent  of  this  report  is  to  inform  prospective  users  about  the  growing  power  and 
potential  of  modeling  and  simulation  for  training  applications.  This  report  also  provides 
information  about  estimating  potential  benefits  of  certain  modeling  and  simulation  tools. 

The  work  effort  was  accomplished  as  part  of  the  aircrew  training  research  program 
at  AL/HRA,  Williams  Gateway  Airport,  Mesa,  A Z,  Colonel  Lynn  A.  Carroll,  Division 
Chief,  and  Dr.  Dee  H.  Andrews,  Technical  Director.  The  research  effort  was 
accomplished  under  Work  Unit  1 123-B2-13,  Unit  Level  Training  Research  Applications 
(ULTRA),  Bemell  J.  Edwards,  project  scientist. 
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POTENTIAL  MODELING  AND  SIMULATION  CONTRIBUTIONS 
TO  AIR  EDUCATION  AND  TRAINING  COMMAND  FLYING  TRAINING: 
SPECIALIZED  UNDERGRADUATE  PILOT  TRAINING 


INTRODUCTION 

Iir  the  spring  of  1994  the  Air  Education  and  Training  Command  (AETC/XOR) 
requested  that  the  Aircrew  Training  Research  Division  of  the  Armstrong  Laboratory 
(AL/HRA)  conduct  an  evaluation  of  AETC's  flying  training  programs.  The  purpose  of  the 
evaluation  was  to  identify  current  and  future  training  challenges/problems  that  might  be 
solvable  via  modeling  and  simulation  (M&S).  This  report  describes  the  goals,  processes, 
and  results  of  the  first  phase  of  that  effort  which  was  directed  toward  an  evaluation  of 
Specialized  Undergraduate  Pilot  Training  (SUPT). 

Modeling  and  simulation  advances  have  been  accelerating  over  the  last  several 
years  so  that  M&S  now  is  an  integral  part  of  many  Air  Force  functions  and  activities. 
These  advances  have  been  made  possible  largely  because  of  dramatic  increases  in 
affordable  microprocessor  capability.  Until  recently,  modeling  and  simulating  even  simple 
processes  has  been  too  expensive  to  be  practical,  but  advances  in  computer  software  now 
make  complex  M&S  systems  affordable  and  effective  for  Air  Force  training. 

While  modeling  and  simulation  technologies  have  obviously  influenced  Air  Force 
training,  it  is  clear  that  much  remains  to  be  done  to  realize  a  remarkable  array  of  potential 
benefits.  As  the  Air  Force  looks  to  the  21st  century,  we  believe  that  M&S  will  graduate 
from  an  adjunct  role  to  a  primary  role  in  training.  Indeed,  M&S  contributions  are 
envisioned  in  all  phases  of  aircrew  ground  training  as  well  as  aircraft  flight  training. 

The  ideas  encapsulated  in  this  report  are  not  all  necessarily  new.  An  Air  Force- 
commissioned  study  from  1971,  summarized  below,  introduced  a  number  of  the  training 
concepts  that  the  Armstrong  Laboratory  investigative  team  expanded  upon  in  the  present 
review.  A  few  of  the  1971  concepts  were  adopted  in  SUPT  training,  but  most  have  not 
become  a  permanent  part  of  the  program.  In  large  measure,  the  delay  in  implementing  these 
otherwise  valuable  concepts  has  been  due  to  a  lack  of  sufficiently  advanced  M&S  tools. 
However,  we  now  have  necessary  technology  to  enable  implementation  of  many  farsighted 
concepts  for  Air  Force  flying  training. 

Literature  Review 

A  review  of  literature  pertinent  to  undergraduate  pilot  training  (UPT)  is  included  in 
the  present  report.  The  review  was  necessary  as  background  information  for  investigators. 

Current  literature  from  AETC  describing  SUPT  programs,  plans,  policies,  and  practices 
was  the  starting  point  for  the  literature  review.  We  reviewed  current  SUPT  course  syllabi, 
training  management  documents,  pilot  instructor  training  (PIT)  course  documents,  and 
SUPT  instructor  techniques  manuals.  Of  special  interest  was  the  AETC  Force 
Modernization  Planning  document  entitled  the  "AETC  Flying  Training  Mission  Area 
Plan."  This  plan  was  examined  and  its  implications  for  the  present  research  effort  are 
discussed  below. 
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In  addition  to  the  above  literature,  we  included  a  survey  of  SUPT-related  training 
research  and  development  in  this  review.  We  found  AL/HRA's  technical  library  to  be  a 
valuable  source  of  data  in  tracing  significant  aspects  of  SUPT-related  technology 
development.  During  the  1970s,  AL/HRA  (then  the  Flying  Training  Division  of  the  Air 
Force  Human  Resources  Laboratory)  conducted  a  number  of  research  efforts  in  support  of 
UPT.  In  reviewing  these  research  reports,  we  found  many  that  were  relevant  to  the 
current  modeling  and  simulation  effort.  Some  of  these  research  efforts  are  cited  below. 

Several  projects  examined  issues  relative  to  syllabus  content  and  training  tasks 
(Brown,  Mullen  &  Rust,  1975;  Brown  &  Rust,  1975;  Smith  &  Flexman,  1972;  Weyer  & 
Fuller,  1976,  1977).  Several  dealt  with  multimedia  training  methods  (Baer,  1972;  Baer  & 
Beggerly,  1973,  McCombs,  Marco,  Sprouls,  Eschenbrenner,  &  Reid,  1973).  Study  of  the 
application  of  cognitive  methods  for  enhancing  portions  of  T-37  training  included  work  by 
Smith,  Waters,  and  Edwards  (1975)  and  Crosby  (1977).  A  number  of  studies  were 
devoted  to  the  evaluation  of  various  aircrew  ground  training  devices  (Wood,  Hagin, 
O’Connor,  &  Myers,  1972;  Woodruff  &  Hagin,  1973;  Woodruff  &  Smith,  1974; 
Woodruff,  Smith  &  Morris,  1972,  1974).  Several  reports  dealt  with  training  analysis, 
design  and  methods  issues  (Baum,  Smith,  &  Goebel,  1972;  Cyrus  &  Woodruff,  1974; 
Leshowitz,  Parkinson,  &  Waag,  1974;  Meyer,  Laveson,  Weisman,  &  Eddowes,  1974; 
Miller,  Swink,  &  McKenzie,  1978;  Tyler,  McFadden,  Eddowes,  &  Fuller,  1976). 

Perhaps  most  relevant  to  current  interests  was  research  dealing  with  simulator 
training  effectiveness.  The  primary  research  vehicle  for  these  studies  was  a  unique  testbed 
device  developed  at  the  laboratory-the  Advanced  Simulator  for  Undergraduate  Pilot 
Training  (ASUPT).  Results  from  projects  using  ASUPT  played  a  major  role  in 
determining  the  application  and  impact  of  simulators  in  pilot  training.  Some  of  these 
studies  included  the  following  topics:  Simulation  issues  and  research  agenda  (Matheny  & 
Gray,  1975);  application  of  full-mission  simulation  in  SUPT  (Woodruff,  Smith,  Fuller,  & 
Weyer,  1976);  simulation  training  features  (Faconti  &  Epps,  1975;  Hughes,  Hannan,,  & 
Jones,  1979);  simulator  visual  and  motion  systems  studies  (Cyrus,  1978;  Martin  & 
Cataneo,  1980;  Martin  &  Waag,  1978;  Nataupsky,  Waag,  Weyer,  McFadden  & 
McDowell,  1979;  Waag,  1981). 

Many  of  the  above  reports  remain  significant  benchmarks  in  the  literature  of  flight 
simulation.  When  published,  they  provided  timely  scientific  evidence  of  the  training  value 
of  simulators. 

In  1971,  the  Aeronautical  Systems  Division  of  Air  Force  Systems  Command 
published  the  findings  of  an  in-depth  study  of  the  UPT  program,  entitled  Future 
Undergraduate  Pilot  Training  (TUPT)  System  Study.  This  was  the  most  comprehensive 
study  of  UPT  reviewed  for  this  current  research  effort.  It  attempted  to  define  the  future 
role  of  UPT  systems  in  the  full  context  of  Air  Force  pilot  training.  Specifically,  the  study 
sought  to  determine  the  characteristics  and  costs  of  alternative  UPT  systems  for  the  1975- 
1990  time  period.  It  addressed  such  broad  issues  as  training  missions  and  requirements, 
facilities,  bases,  personnel,  operational  environments,  and  management  subsystems.  It  also 
examined  curricula/syllabi,  training  methods,  training  equipment  and  aircraft  requirements. 
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A  major  portion  of  the  FUPT  study  was  devoted  to  the  potential  of  emerging 
simulation  technologies.  A  careful  review  of  the  FUPT  study  provided  retrospective 
insights  on  training  issues  of  that  era  compared  to  those  of  the  present.  It  was  possible,  by 
referencing  findings  and  recommendations  of  the  FUPT  report,  to  note  some  of  the 
subsequent  changes  which  actually  occurred  in  the  UPT  program  during  the  1975-1990 
period.  _ 

Some  of  these  changes,  clearly,  were  influenced  by  the  report.  Long-term 
outcomes  reflected  both  accuracies  and  limitations  within  the  study.  For  example,  as 
forecast,  the  impact  of  simulators  in  UPT  and  elsewhere  has  been  substantial,  with 
advances  in  technology  continuing  to  the  present.  Yet,  in  some  ways,  the  potential  of 
technology  has  been  less  than  hoped.  As  previously  observed,  some  of  the  ideas  set  forth 
in  the  report  were  premature  to  the  capabilities  of  technologies  of  that  time.  However, 
another  observation  drawn  from  the  study  was  that  benefits  from  technology  are  usually 
proportional  to  the  soundness  of  its  implementation.  Several  technologies  recommended  in 
the  study  appear  to  have  had  little  success,  not  necessarily  because  of  inherent  limitations, 
but  more  likely  because  of  poor  implementation. 

The  AETC  Flying  Training  Mission  Area  Plan  was  one  of  the  first  documents 
reviewed  for  the  present  study.  As  part  of  Air  Force  Modernization  Planning,  it  was 
developed  with  a  view  toward  investment  strategies  for  training  systems  modernization. 
Specifically,  the  purpose  of  the  plan  is  to  provide  AETC  with  a  vehicle  to  anticipate  and 
overcome  training  deficiencies  thereby  improving  training  effectiveness  and  minimizing 
costs.  In  purpose  and  scope,  the  AETC  Modernization  Plan  bears  some  similarities  to  the 
earlier  FUPT  study.  The  plan  deals  with  a  number  of  broader  issues  in  defining 
modernization  requirements.  This  includes  needs  assessments  with  emphasis  on  AETC’s 
expanded  training  role  in  Combat  Crew  Training  Squadrons  (CCTS)  environments.  Other 
issues  addressed  include  concept  of  operations  (CONOPS),  future  missions  and  training 
tasking,  assessment  of  current  and  future  training  requirements  and  deficiencies, 
infrastructure  deficiencies,  aircraft  modernization,  aircraft  technology  leveraging  strategies, 
management  system  leverage  technologies,  and  ground-based  training  technology 
leveraging  strategies.  These  last  two  categories  were  of  particular  interest  for  purposes  of 
the  present  modeling  and  simulation  study. 

The  plan  listed  a  wide  variety  of  computer-based  technologies  for  modernization  of 
management  programs  and  ground  training  systems.  AETC  will  consider  these  as  potential 
leveraging  technologies  toward  modernization  efforts.  Technologies  thus  identified  appear 
generally  consistent  with  those  considered  in  this  report.  However,  the  (draft)  AETC 
planning  document  we  examined  only  identified  potentially  useful  technologies.  Our 
inference  is  that  analysis  and  application  of  M&S  technologies  were  not  within  the  scope 
of  the  document.  The  Modernization  Plan  does  not  address  modeling  and  simulation 
technology  in  detail,  and  we  hope  that  the  analysis  and  results  of  the  present  study  will 
usefully  extend  understanding  of  how  modem  technology  can  enhance  SUPT  programs. 
Mattoon  (1995)  completed  a  broad  review  of  the  psychology,  education,  and  training 
literature  after  the  present  study  to  provide  additional  support  to  the  M&S  technologies 
proposed  below. 
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Scope  of  Analysis 

The  Simulator  Four-Star  Review  and  the  Modeling  and  Simulation  Four-Star 
Review,  both  held  in  1993,  revealed  the  potential  of  M&S  for  achieving  Air  Force  (AF) 
training  objectives.  The  Air  Force  Chief  of  Staff  recently  made  clear  the  importance  and 
priority  of  modeling  and  simulation  as  keys  to  the  future  of  Air  Force  training  at  all 
levels.  The  Air  Force's  establishment  of  the  Modeling  and  Simulation  Office  at  the  Air 
Staff  demonstrates  the  commitment  and  priority  given  to  modeling  and  simulation 
technologies  for  aircrew  training.  AETC  expects  large-scale  benefits  from  the  infusion  of 
M&S  beginning  in  SUPT  and  extending  to  CCTS  and  continuation  training  environments. 

The  Draft  DoD  Modeling  and  Simulation  Master  Plan  of  30  Sep  94  (DoD  5000.59- 
P)  defines  a  model  as,  "A  physical,  mathematical,  or  otherwise  logical  representation  of  a 
system,  entity,  phenomenon,  or  process."  A  simulation  is,  "A  method  for  implementing  a 
model  over  time." 

In  aircrew  training,  the  use  of  models  is  readily  apparent.  Some  applications 
include  models  of  natural  phenomena,  aerodynamic  equations,  force  cueing,  radar,  and 
weather  systems.  Such  models  may  enable  users  to  study  cause-effect  relationships  in 
order  to  understand  processes  within  complex  systems. 

We  believe  that  modeling  and  simulation  capabilities  will  increase  significantly  as 
aircrew  training  tools.  For  example,  it  will  be  possible  to  accurately  model  student’s 
mental  makeup  and  learning  style  so  that  instructors  can  better  assess  their  training  needs 
and  guide  activities.  In  addition,  we  will  be  able  to  model  many  of  the  dynamic  elements 
within  training  programs  in  order  to  automate  major  portions  of  the  management  process. 

From  a  research  and  development  (R&D)  perspective,  there  is  some  risk  in 
attempting  to  apply  M&S  technologies  to  existing  training  systems  without  undertaking  a 
complete  redesign  of  the  training  system.  The  hazard  is  that  technology  applications  end 
up  treating  symptoms  rather  than  endemic  causes.  The  risk  occurs  because  M&S 
applications  may  not  be  optimal  if  the  goals  and  processes  of  the  whole  system  are  not 
understood.  This  point  of  view  is  well  supported  in  the  training  literature  as  well  as  by 
many  years  of  experience  with  military  training  systems. 

METHOD 

Implicit  in  AETC's  request  for  this  research  was  tacit  agreement  that  a  detailed 
needs  assessment  would  reduce  the  risk  of  M&S  applications  resulting  as  "band-aid” 
solutions.  The  procedures  described  below  were  designed  to  provide  AL/HRA  with 
information  necessary  for  training  system  analysis.  Two  questions  emerged:  First,  "What 
are  the  major  training  challenges  in  SUPT?"  and  second,  "What  modeling  and  simulation 
technologies  are  available  [or  in  development]  which  can  help  meet  these  challenges?"  To 
address  these  questions,  data  gathering  and  analytical  procedures  were  used  in  combination 
with  consultation  with  subject-matter  experts  and  "think  tank"  techniques. 

First,  we  formed  a  team  of  investigators  within  the  laboratory.  This  research  team 
was  composed  of  Air  Force  pilots,  research  psychologists,  and  training  systems  engineers. 
Operations  researchers,  educational  technologists,  and  former  military  pilots  were  used  on 
a  consulting  basis  from  outside  the  laboratory.  A  suitable  approach  to  identifying  training 
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a  consulting  basis  from  outside  the  laboratory.  A  suitable  approach  to  identifying  training 
problems  was  required.  It  was  also  apparent  that  some  types  of  analysis,  such  as  task 
analysis,  would  be  inappropriate.  The  objective  was  to  identify  both  problems  and 
solutions.  After  due  consideration,  we  selected  needs  assessment  procedures  recommended 
by  Kaufman  (1991).  In  connection  with  the  identification  of  needs  and  their  meaningful 
solutions, ~we  believe  the  methodology  described  in  this  report  accomplished  the  intended 
purposes. 

Three  conditions  must  be  satisfied  in  a  successful  needs  assessment  study.  First, 
the  appropriate  form  and  substance  of  inquiry  must  be  determined  and  cast  into  a  suitable 
vehicle.  Second,  the  inquiry  must  be  addressed  to  the  proper  subject  population.  Third, 
data  resulting  from  the  inquiry  must  be  correctly  interpreted. 

Questionnaire  Development.  Investigators  developed  a  questionnaire  with  the 
objective  of  producing  a  valid  vehicle  for  the  inquiry.  Psychologists  and  pilots  required 
several  revision  cycles  to  agree  on  content.  The  questionnaire  was  then  circulated  to 
managers  and  qualified  pilots  for  critique  and  comment.  Reviewers  included  training 
managers  at  the  56th  Training  Squadron  at  Luke  AFB,  A Z,  and  AETC/XOR  at  Randolph 
AFB,  TX.  Using  inputs  from  these  sources,  investigators  again  revised  the  questionnaire. 
It  was  then  distributed  to  selected  personnel  at  the  64th  Operations  Support  Squadron 
(OSS)  at  Reese  AFB,  TX  for  validation  and  final  revision. 

A  structured  interview  technique  using  questionnaire  content  was  selected  as  the 
most  efficient  method  for  obtaining  information  about  the  SUPT  program.  Currently 
assigned  SUPT  instructor  pilots  at  Reese  AFB  were  chosen  as  the  target  population.  The 
Reese  site  was  used  because  it  was  the  only  SUPT  base  at  the  time  with  an  active  T-l 
program  in  addition  to  the  T-37  and  T-38  programs.  The  choice  of  the  instructor  pilots  was 
based  on  the  rationale  that  they  would  be  the  most  knowledgeable  and  direct  source  of 
information  on  the  day-to-day  operational  training  problems  within  SUPT.  A  copy  of  the 
questionnaire  materials  is  contained  in  Appendix  A. 

Preliminary  Technology  Identification.  Concurrently  with  development  of  the 
questionnaire,  investigators  prepared  a  preliminary  list  of  candidate  modeling  and 
simulation  technologies.  This  list  was  distributed  to  training  experts  at  AL/HRA  for 
review.  The  preliminary  list  was  revised  based  on  the  opinions  of  reviewers. 

Data  Collection.  The  research  team  conducted  instructor  pilot  interviews  on-site 
at  the  64th  OSS  in  June  1994.  Twenty-eight  instructor  pilots  assigned  to  the  squadron 
participated,  most  of  whom  were  captains  and  majors  (three  of  the  interviewees  were 
contract  simulator  instructors  for  the  T-l  program).  Pilots  were  interviewed  in  groups  of 
two  or  three.  Two  interviewer  teams,  each  comprised  of  a  research  pilot  (who  had  prior 
instructor  pilot  experience)  and  two  research  psychologists,  conducted  interviews.  The 
research  pilot  asked  the  questions  and  led  the  discussion.  Conversations  were  primarily 
pilot-to-pilot  which  helped  keep  the  dialog  spontaneous  and  facilitated  clear 
communication.  Psychologists  wrote  the  instructors’  responses  to  questions  on  blank 
copies  of  the  questionnaire.  We  made  audio  recordings  of  interviews  (with  the 
interviewees'  knowledge)  as  a  backup  source  of  information.  A  senior  psychologist  also 
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interviewed  selected  members  of  squadron  management.  These  interviews  were  conducted 
apart  from  the  instructor  pilot  interviews  using  a  separate  set  of  questions. 

Problem  Identification  and  Analysis.  Following  data  collection,  members  of 
each  research  team  examined  the  completed  questionnaires  and  compiled  response 
information.  Psychologists  first  reviewed  and  tabulated  data  for.  each  of  the  groups  they 
had  interviewed.  They  then  worked  jointly  in  categorizing  and  analyzing  response  data. 
Comments  by  pilots  were  organized  by  assigned  type  of  aircraft  (T-37,  T-38  or  T-l).  In 
tabulating  questionnaire  data,  psychologists  used  audio  recordings  as  needed  to  clarify 
written  responses.  Pilot  responses  were  categorized  by  areas  of  training  reported  as  being 
problematic.  Several  categories  emerged  from  this  sorting  process.  We  found  that 
instructors  tended  to  describe  training  problems  in  three  basic  dimensions:  (a)  difficulties 
encountered  by  students  in  attempting  to  learn,  (b)  difficulties  encountered  by  instructors  in 
attempting  to  teach  students,  and/or  (c)  factors  within  the  training  program  itself  which 
cause  or  contribute  to  problems  of  training. 

Investigators  pursued  a  second  level  of  analysis  which  involved  referencing 
categorized  problems  to  the  formal  training  program  (syllabus  objectives  and  content).  We 
reviewed  training  objectives  to  determine  specific  points  at  which  problems  occur  or  may 
occur.  The  effects  of  each  problem  on  stated  training  objectives  were  also  analyzed.  To 
validate  this  analysis,  a  panel  of  pilots  with  prior  SUPT  instructor  experience  examined  the 
relationship  of  problems  to  syllabus  content.  Pilots  confirmed  the  validity  of  this  analysis. 

A  third  level  of  analysis  involved  in-depth  examination  of  causal  factors. 
Psychologists  and  research  pilots  jointly  probed  a  variety  issues  for  each  of  the  problem 
areas.  The  causal  analysis  provided  perhaps  the  most  useful  basis  for  identifying  potential 
technology  solutions.  A  natural  tendency  during  these  discussions  was  the  extrapolation  of 
"fixes."  However,  so  that  the  problem  definition  process  would  remain  separate  from  the 
solution  development  process,  systematic  consideration  of  solutions  was  reserved  until  the 
entire  problem  analysis  phase  was  concluded.  Problem  analysis  was  pursued  until  all  team 
members  were  satisfied  with  the  definitions,  descriptions,  and  contributing  causes  for  each 
problem  area.  We  reviewed  written  descriptions  to  ensure  documentation  was  accurate  and 
sufficiently  detailed  to  communicate  the  findings. 

We  identified  six  primary  challenge/problems  areas:  position  identification,  VHF 
Omnidirectional  Range/Distance  Measurement  Equipment  (VOR/DME)  fix-to-fix 
navigation,  overhead  landing  pattern,  formation  flight,  low  attitude  training  and  instructor 
pilot  continuation  training.  Each  of  these  areas  is  described  in  detail  later  in  this  report. 

Identification  of  Technology  Solutions.  Using  the  preliminary  list  of  M&S 
technologies  (described  above),  investigators  considered  various  technology  options  as 
solutions  for  each  problem  area.  We  selected  the  focus  group  technique  as  the  process  of 
greatest  advantage  for  this  phase  of  the  effort.  The  focus  group  method  provides  an  avenue 
toward  group  consensus.  During  group  sessions,  team  members  explored  a  wide  range  of 
ideas  about  technology.  At  the  outset,  several  procedures  and  rules  were  invoked  to  control 
the  discussions.  A  member  of  the  team,  with  experience  in  focus  group  methods,  organized 
and  led  the  discussions. 
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The  product  of  focus  group  efforts  was  a  detailed  set  of  technology  descriptions 
matching  the  identified  problems.  Some  of  the  solutions  were  found  applicable  to  more 
than  one  problem  area.  In  describing  technologies,  we  also  decided  that  visual  media 
(videotape)  would  help  clarify  the  characteristics  and  capabilities  of  some  modeling  and 
simulation  concepts.  Focus  group  discussions  required  approximately  five  working  days  to 
complete.' 

Two  groups  of  experts  at  the  laboratory  were  asked  to  evaluate  the  matching  of 
problems  with  proposed  solution  technologies.  Briefings  were  presented;  first,  to  a  panel 
of  in-house  engineers  and  pilots;  and  second,  to  a  panel  of  research  psychologists. 
Suggestions  from  these  groups  were  integrated  with  descriptive  materials  to  improve  the 
completeness  and  accuracy  of  technology  descriptions.  The  content  of  the  briefing  and 
support  media  materials  was  revised  for  final  presentation 

The  next  step  was  to  present  the  results  of  the  team's  findings  to  the  SUPT 
community.  This  was  accomplished  in  two  steps.  First,  four  instructor  pilots  from  the 
64th  OSS  at  Reese  AFB  were  invited  to  come  to  AL/HRA  to  evaluate  results  of  the 
problem-solution  matching  process.  Pilots  were  given  demonstrations  and  first-hand 
experience  with  some  of  the  technologies.  They  generally  were  able  to  validate  the 
matching  of  technologies  with  the  training  problems.  They  also  identified  other  training 
needs  which  had  not  been  emphasized  in  the  June  group  interviews.  These  included  the 
importance  of  the  debriefing  in  the  pilot  training  process  and  the  need  for  the  best  possible 
media/methods  to  support  debriefings.  The  instructors  also  indicated  that  low-altitude 
training  is  a  challenge  in  SUPT  and  suggested  several  technology  approaches.  We  added 
this  topic  to  the  original  list  of  problems.  The  process  of  analyzing  low-altitude  training 
and  identifying  solutions  followed  the  same  analytical  procedures  previously  described. 

An  additional  short  questionnaire  was  developed  as  a  result  of  the  inputs  from  the 
four  instructors.  Its  purposes  were:  (a)  to  allow  pilots  to  verify  results  of  the  analysis  and 
description  of  training  problems,  and  (b)  to  obtain  estimates  from  instructor  pilots  as  to  the 
usefulness  of  the  various  technology  solutions  for  problems  identified. 

As  the  second  and  completion  step  in  this  part  of  the  research,  members  of  the 
research  team  returned  to  the  64th  OSS  at  Reese  AFB  to  brief  SUPT  personnel  on  the 
study  findings.  Fourteen  instructor  pilots,  most  of  whom  had  participated  in  the  June 
interviews,  were  present  for  the  briefing.  Various  research  team  members  briefed  various 
aspects  of  the  study.  Video  clips  were  used  to  demonstrate  proposed  technologies.  Using 
the  questionnaire,  instructors  rated  the  degree  of  seriousness  of  each  training  problem  area 
identified  in  the  study  posed  to  the  SUPT  program.  They  also  rated  the  usefulness  of  each 
proposed  technology  as  a  training  problem  solution.  The  results  of  this  questionnaire  are 
presented  in  the  Results  section  below  (see  Instructor  Feedback  on  Analysis  and  Results). 


RESULTS 

This  section  describes  the  major  findings  on  training  problems  and  challenges 
identified  by  this  research  effort.  Results  are  described  under  two  major  headings  in  this 
section:  Description  of  Training  Problems  Areas,  and  Proposed  Modeling  and  Simulation 
Technology  Solutions. 
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Description  of  Training  Problem  Areas 

Position  Interpretation.  Instructor  pilots  (IPs)  report  that  position  interpretation  is 
one  of  the  most  pervasive  problems  in  SUPT.  This  is  a  basic  flying  skill  which  enables 
students  to  interpret  navigation  instrumentation  to  know  where  they  are  at  any  given  point 
in  the  flight.  Instructor  pilots  indicate  that  this  is  a  major  training  problem  for  the 
following  reasons: 

1.  The  T-37  is  poorly  instrumented.  As  a  1950’s-era  aircraft,  its  navigation 
aids  are  outmoded,  and  the  Tactical  Aid  to  Navigation  (TACAN)  and  VOR/DME  are  no 
longer  used  outside  of  SUPT.  The  poor  design  of  aircraft  instrumentation  increases  the 
difficulty  of  this  task  for  the  student.  It  is  hard  for  students  to  translate  abstract  data  from 
the  instruments  into  real-world  geometry.  While  improved  instrumentation  is  expected  in 
the  Joint  Service  Pilot  Aircrew  Training  System  (JPATS),  that  new  aircraft  will  not  be 
operational  for  some  time.  Thus,  the  position  interpretation  problem  will  continue  for  the 
remaining  service  life  of  the  T-37.  IPs  also  indicated  that  position  interpretation  persists 
as  something  of  a  problem  in  T-38  and  T-l  training.  Position  interpretation  is  a  subskill  of 
situational  awareness  and,  therefore,  a  fundamental  concern  in  developing  basic  piloting 
skills. 

2.  This  problem  is  compounded  during  flying  because  of  the  number  of 
concurrent  tasks  which  compete  for  the  pilot’s  attention.  Novice  students,  who  have  not 
yet  learned  to  prioritize  and  manage  multiple  tasks,  tend  to  become  overloaded  mentally 
so  that  awareness  of  correct  position  may  be  lost.  However,  position  interpretation  is  a 
skill  that  must  be  acquired  and  perfected  regardless  of  the  quality  of  the  cockpit 
instruments. 

3.  The  complexity  of  position  interpretation  is  also  seen  from  the  instructor's 
viewpoint.  During  interviews,  IPs  expressed  concern  that  some  instructors,  themselves, 
may  not  have  radial  geometry  concepts  firmly  set  in  their  own  minds.  The  concern  seems 
to  be  that  an  instructor  may  execute  the  task  correctly,  but  only  at  a  procedural  or 
mechanical  level.  We  interpret  this  to  mean  that  while  some  instructors  may  be  able  to 
perform  correctly,  they  may  be  unable  to  teach  students  what  they  are  doing,  why  they  are 
doing  it,  or  to  convey  concepts  underlying  the  procedures. 

VOR/DMF.  FiTc-tn-FiY  Navigation.  Position  interpretation  is  an  enabling  skill  for 
performing  fix-to-fix  navigation.  Like  position  interpretation,  fix-to-fix  navigation  is  a 
three-dimensional  (3D)  concept  that  is  difficult  to  learn  on  the  ground.  Chair  flying 
(mental  practice)  is  often  used  but  it  is  difficult  to  attempt  mental  practice  without  being 
able  to  visualize  complex  instrument  readings.  The  ability  to  visualize  comes  only  after 
considerable  practice  in  the  aircraft.  In  addition,  IPs  are  concerned  that  the  instrument 
regulation,  Air  Force  Manual  51-37.  is  incomplete  due  to  its  limited  textual  and  graphical 
content.  Because  presentations  on  paper  are  static,  they  do  not  convey  the  dynamic  nature 
of  the  task  and  the  environment  in  which  fix-to-fix  is  performed.  IPs  are  concerned  that 
there  are  few  opportunities  for  students  to  practice  fix-to-fix  and  receive  meaningful 
feedback.  When  students  are  finally  able  to  hold  a  mental  picture  of  the  task  sufficient  for 
mental  practice,  they  have  largely  learned  the  task  in  the  aircraft. 
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During  aircraft  training,  students  easily  become  task-saturated  and  may  miss  verbal 
instructions.  Students  at  this  stage  are  still  trying  to  perfect  position  interpretation  skills 
and,  thus,  cannot  practice  fix-to-fix  effectively  without  first  knowing  where  they  are. 
Getting  off  course  is  a  common  error  which  only  compounds  the  problem.  Time  required 
to  reach  the  final  fix  may  be  so  long  that  the  IP  finds  it  difficult  to  revert  to  any  initial 
problems.-  Finally,  it  is  difficult  for  IPs  to  assess  the  proficiency  of  students’  fix-to-fix 
attempts.  The  initial  turn  is  hard  to  evaluate  “on  the  fly.”  The  number  of  course 
corrections  that  need  to  be  made  tend  to  mask  the  students’  true  proficiency. 

Overhead  Traffic  Pattern.  SUPT  instructors  consistently  identified  the  T-37 
overhead  traffic  pattern  as  one  of  the  most  challenging  training  tasks  in  the  entire  syllabus. 
Several  reasons  account  for  its  difficulty,  for  both  students  and  instructors.  Because 
students  must  learn  to  solo  quite  early  in  the  T-37  phase,  they  need  to  acquire  landing  skills 
first.  This  creates  a  “front-loaded”  training  demand  on  the  student,  the  instructor,  and  the 
system. 

Academic  training  plays  an  important  part  in  familiarizing  students  with  this  task. 
However,  for  the  student,  the  associated  cognitive  and  procedural  skills  are  difficult  to 
acquire  short  of  training  in  the  aircraft.  Being  able  to  practice  cognitive  and  procedural 
skills  in  a  ground  training  device  would  improve  the  rate  of  skill  transfer  to  the  aircraft. 
However,  a  learning  gap  between  academics  and  flight  training  is  very  evident  for  this  task. 
Tools  are  also  needed  to  permit  students  to  develop  critical  procedural  repertoires  prior  to 
entering  the  aircraft. 

The  full  impact  of  this  task  "comes  home"  when  the  students  first  try  to  land  the 
aircraft.  Students  find  it  difficult  to  execute  procedures  in  proper  order  while  maintaining 
the  correct  flight  envelope.  Events  transpire  quickly,  the  student  becomes  mentally 
overloaded,  "gets  behind  the  jet,"  and  the  instructor  must  intervene. 

From  an  instructor  perspective,  the  overhead  traffic  pattern  is  a  challenging  task  to 
demonstrate  and  teach  to  students.  Instructors  need  ground  training  tools  which  can 
convey  essential  spatial-geometric  aspects  of  the  task.  Present  simulators  limit  the 
instructor's  ability  to  demonstrate  model  landing  patterns  and  to  show  common  errors  and 
corrective  actions. 

Training  in  the  aircraft  challenges  instructors.  Aircraft  resources  are  limited,  so 
aircraft  sorties  allocated  to  landing  training  are  also  limited.  The  critical  nature  of  the  task 
tends  to  increase  stress  on  the  instructor,  particularly  in  cases  of  marginal  student 
performance. 

The  training  challenge  continues  after  each  training  flight.  Instructors  indicate  that 
the  debrief  is  potentially  a  very  powerful  training  opportunity,  but  a  lack  of  effective 
debriefing  tools  limits  the  instructor's  ability  to  give  feedback  to  the  student.  During  flight, 
the  IP  attempts  to  collect  as  much  information  as  possible,  but  monitoring  the  student  and 
the  aircraft  leaves  little  opportunity  to  write  notes.  Instructors  rely  largely  on  mental  recall 
for  debriefings.  No  technology  is  currently  available  for  capturing  and  replaying  SUPT 
aircraft  sorties.  Such  a  capability  would  allow  instructors  to  provide  students  with  much 
more  precise  performance  feedback.  This,  in  turn,  would  help  optimize  the  value  of 
debriefings  as  part  of  the  learning  process. 
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In  many  ways,  the  overhead  landing  pattern  is  a  cross  section  of  the  entire  syllabus. 

It  requires  a  composite  of  exacting  flying  skills  executed  in  rapid  sequence.  Current 
training  challenges  to  both  instructors  and  students  could  be  largely  eliminated  with 
advanced  modeling  and  simulation  techniques  that  could  capture  and  replay  selected  flight 
sequences. 

Formation  Flight.  Learning  to  maneuver  a  high  performance  aircraft  in  close 
proximity  to  another  aircraft  presents  a  training  challenge  unique  to  military  pilot  training. 
At  the  outset,  instructors  must  have  considerable  confidence  in  students'  piloting  ability, 
since  students  have  not  yet  attempted  this  level  of  flying.  In  formation  flight  training, 
students  must  first  learn  to  perform  takeoffs,  climbs,  turns,  mild  maneuvering,  instrument 
approaches  and  landings  as  two-ship  operations.  The  student  must  then  become  proficient 
in  separating  from,  and  rejoining,  formations  (break-ups  and  rendezvous).  All  formation 
flight  maneuvers  require  students  to  expand  situational  awareness  to  include  other  aircraft. 
Essential  new  skills  include  determination  of  angle  off,  bearing  line,  and  closure  rates,  all 
of  which  demand  rapid,  accurate  spatial  judgments.  Students  learn  a  large  repertoire  of 
visual  cues  associated  with  precision  aircraft  control.  The  exacting  character  of  formation 
flight  requires  an  environment  in  which  students  have  ample  opportunity  for  effective 
practice.  Yet,  except  for  the  aircraft  itself,  no  practice  device  is  available  for  this  task. 
Particularly  for  close  formation,  visual  discrimination  of  surface  details  of  the  lead  aircraft, 
used  in  judging  distance  and  closure  rates,  are  critical  cues.  But  the  present  simulators 
lack  sufficient  visual  fidelity  to  display  these  cues.  Instructors  familiarize  students  as 
much  as  possible  using  available  teaching  tools  such  as  diagrams,  verbal  descriptions  and 
stick  models.  The  net  result  of  inadequate  ground  training  is  the  consumption  of  large 
amounts  of  jet  fuel  while  students  develop  needed  skills  in  the  aircraft  itself.  Thus, 
formation  flight  is  another  area  of  training  that  would  benefit  dramatically  from  simulation 
modernization. 

Low- Altitude  Flight.  Low-altitude  flying  is  a  critical  task  involving  knowledge 
and  skills  that  differ  from  other  flying  activities.  Evidence  shows  the  vast  majority  of  low- 
altitude  flight  accidents  are  caused  by  poor  task  management.  Experts  in  this  flight  regime 
have  developed  a  task  management  concept  to  reduce  its  risks.  The  concept  organizes  low- 
altitude  flight  into  three  piloting  activities  or  priorities  as  follows: 

1.  The  first  priority,  obviously,  is  to  maintain  enough  altitude  to  avoid  hitting  the 
ground.  Thus,  Terrain  Clearance  Tasks  (TCTs)  are  those  actions  the  pilot  must  take  to 
maintain  a  safe  margin  of  altitude  at  all  times. 

2.  Second  priority  tasks  are  those  the  pilot  must  accomplish  to  complete  the 
mission.  These  are  called  Critical  Tasks  (CTs).  They  include  navigation,  communication, 
and  other  requirements. 

3.  Finally,  in  third  priority  are  Non-Critical  Tasks  (NCTs)  that  the  pilot  can 
accomplish  as  time  permits  during  periods  not  demanding  attention  by  TCTs  and  CTs. 

Thus  conceived,  low-altitude  flight  becomes  an  activity  in  which  the  pilot 
continuously  evaluates  aircraft  status  and  mission  demands  in  order  to  allocate  attention  to 
the  task  of  immediate  priority. 
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Safe  low-altitude  flying  requires: 

1.  Vector  Control— assessing  and  modifying  the  aircraft  vector,  elevation,  azimuth, 
and  velocity  relative  to  terrain; 

2.  Above  ground  level  (AGL)  control-assessing  and  modifying  aircraft  AGL  in 
relation  to  terrain;  and 

3.  Time  Control— mentally  controlling  the  frequency  and  duration  of  mission 
cross-check  time  relative  to  the  existing  TCT  vector  and  AGL  control  requirements. 

CTs  and  NCTs  for  SUPT  would  be  simplified  subsets  of  the  components  of  tactical 
fighter  low-altitude  training.  TCTs  are  similar  in  nature,  although  SUPT  altitude 
minimums  are  substantially  higher  than  those  in  fighter  training.  A  proposed  set  of  CT  and 
NCT  components  in  SUPT  include: 

1 .  Situation  awareness— sorting  and  sequencing  mission  tasks  and  anticipating  the 
near-term  flight  situation  in  order  to  make  safe  and  effective  aircraft-control  decisions; 

2.  Navigation— interpreting  and  using  spatial  and  navigation  displays,  controlling 
closure  time  and  distance,  adjusting  heading,  controlling  aircraft  orientation  (yaw,  pitch, 
and  roll),  altitude,  and  using  ground  culture  and  dynamic  visual  information;  and 

3.  Performing  command,  control,  and  communication  (C3)~ radio  communications 
between  aircraft  and  formation  aircrew  members  and  tower. 

Low-altitude  flying  involves  the  ability  to  recognize  critical  visual  cues  quickly  and 
accurately  under  heavy  task-loading  conditions.  The  current  method  of  training  involves 
textbook  study  and  classroom  instruction,  chair  flying,  and  training  in  the  aircraft.  Students 
are  under  close  observation  of  an  IP  during  initial  low-altitude  flight  training. 

Although  the  primary  training  environment  is  the  aircraft,  in  practice,  low-altitude 
flight  training  sorties  may  be  delayed,  or  too  few  in  number  due  to  adverse  weather  or 
aircraft  nonavailability.  These  eventualities  may  reduce  available  training  time  and 
ultimately  affect  student  flying  proficiency. 

The  current  problem  in  low-altitude  training  arises  largely  from  simulator  visual 
system  inadequacies.  As  with  formation  flight  training,  safe  low-altitude  flight  is  heavily 
dependent  upon  the  pilot's  ability  to  discriminate  visual  features  of  ground  objects  and 
texture  details.  The  Operational  Flight  Trainer  (OFT)  for  the  T-37,  T-38,  and  T-l  lack 
visual  systems  with  an  adequate  fidelity  for  this  task.  Low-altitude  training  simulation 
requires  high-speed  (e.g.,  60  Hz  update  rate),  high-resolution  visual  display  of  ground  cues 
(e.g.,  trees,  riverbeds,  buildings)  at  different  altitudes,  air  speeds,  and  angles  of  view. 
Fortunately,  simulator  visual  systems  state  of  the  art  is  now  capable  of  providing  these 
high  levels  of  fidelity  at  reasonable  cost.  Given  the  benefits  of  modem  simulators,  SUPT 
students  would  be  able  to  develop  many  low-altitude  flying  skills  prior  to  aircraft  training. 
This  would  significantly  increase  flight  safety  and  reduce  the  time  and  cost  of  this 
training. 

Instructor  Continuation  Training.  The  current  instructor  pilot  training  program 
requires  that  each  IP  spend  eight  weeks  at  the  PIT  course  at  Randolph  AFB.  During  this 
course,  each  IP  receives  training  on  how  to  instruct,  both  in  the  classroom  and/or  in  the 
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aircraft.  Some  of  this  training  is  intended  to  give  the  IP  a  sensitivity  for  instructional 
principles  relevant  to  problems  instructors  will  face  with  students  in  the  training  squadron. 
These  principles  as  embodied  in  Systems  Approach  to  Training  (SAT)  include: 

-  defining  training  tasks 

-  demonstrating  flying  procedures  and  skills 

-  understanding  internal  and  external  conditions  of  learning 

-  evaluating  trainee  progress 

-  diagnosing  learning  problems 

-  providing  feedback  at  an  appropriate  time  in  an  effective  manner 

However,  due  to  the  brief  span  of  the  PIT  course,  only  a  cursory  treatment  of  these 
principles  is  provided.  It  also  may  be  that  the  present  PIT  treatment  of  instructional 
principles  is  not  sufficiently  specific  or  focused  for  effective  application  by  new  instructors 
when  they  reach  the  flightline.  Also,  because  of  limited  time  in  the  course,  new  IPs  may 
tend  to  devote  more  time  to  re-familiarization  with  their  assigned  training  aircraft  (T-37, 
T-38,  or  T-l)  than  to  developing  training  skills  and  strategies.  Since  AETC  already  has  a 
comprehensive  mechanism  for  developing  instructional  programs,  it  is  not  necessary  for 
each  IP  to  have  a  deep  understanding  of  instructional  principles  required  for  syllabus 
development,  but  they  should  be  well  grounded  in  basic  and  mid-level  instructional 
principles  and  diagnostic/prescriptive  techniques  to  deal  effectively  with  student  learning 
problems. 

Our  analysis,  based  on  interviews  with  PIT  personnel  and  instructors,  revealed  that 
IPs  presently  receive  insufficient  training  on  the  application  of  instructional  principles  and 
teaching  techniques.  During  the  interviews  we  conducted,  instructors  almost  always 
characterized  student  learning  problems  in  terms  of  the  inability  to  perform  flying  tasks 
rather  than  in  terms  of  learning  processes  or  instructional  needs.  At  times,  instructors 
seemed  hard-pressed  to  identify  specific  information  to  give  students  which  would  help  the 
students  understand  why  they  were  having  problems.  In  prescribing  activities  and 
exercises  to  aid  students,  instructors  tended  to  repeat  generic  “fixes”  such  as  more  chair 
flying,  better  debriefings,  more  practice,  etc.,  for  a  range  of  problems.  The  net  effect  of 
not  receiving  more  by  way  of  teaching  skills  in  PIT  is  an  IP  who  arrives  at  the  SUPT 
squadron  with  a  rather  sparse  instructional  tool  kit. 

Also,  there  appears  to  be  no  formal  continuation  training  for  IPs  once  they  reach 
their  training  squadrons.  IPs  seem  to  rely  on  the  training  methods  they  themselves 
observed  as  SUPT  students.  Squadron  leaders  and  fellow  IPs  routinely  share  experiences 
and  insights,  but  this  probably  does  not  enable  IPs  to  train  at  their  highest  potential,  nor 
will  such  sharing  of  training  wisdom  always  be  complete,  consistent,  or  accurate. 

For  these  reasons,  systematic  continuation  training  for  IPs  in  the  squadrons  is 
needed.  Time  is  an  obstacle  for  such  training.  IPs  have  more  than  enough  to  do,  but  we 
believe  that  even  two  hours  per  month  for  this  kind  of  training  would  increase  IP 
effectiveness  thereby  increasing  the  efficiency  of  the  entire  SUPT  program.  Several  M&S 
technologies  described  later  in  this  report  could  be  used  to  help  deliver  IP  continuation 
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training.  Technology  could  do  much  to  decrease  the  time  burden  by  maximizing  use  of 
discretionary  time.  Some  of  this  training  might  occur  in  a  classroom;  some  might  be 
delivered  via  personal  computers-content  of  such  training  is  critical.  If  it  is  noticeably 
helpful  in  reducing  the  IP's  workload  or  solving  student  training  problems,  they  will  use  it. 


Proposed  Modeling  and  Simulation  Technology  Solutions 


This  section  continues  the  reporting  of  our  research  results.  It  describes  the 
modeling  and  simulation  technologies  identified  as  the  result  of  the  analysis  process 
reported  earlier.  Each  section  below  describes  the  characteristics  and  functions  of  the 
recommended  technologies,  plus  an  indication  of  the  training  problems  they  will  address. 
Estimated  costs  for  AETC  implementation  of  these  technologies  are  included  in  the 
roadmaps  presented  later  in  this  report. 

SUPT  Hub  Computer  System.  This  system  will  function  as  the  “brain”  of 
modeling  and  simulation  training  applications  proposed  in  this  report.  It  will  contain  and 
manage  all  data,  models,  and  media-associated  activities.  This  will  extend  to  simulation 
models  of  the  environment  not  contained  and  directly  managed  by  individual  simulators 
and  training  devices.  The  database  will  contain  libraries  of  training  scenarios  and  will 
contain  all  training-relevant  student  data.  The  hub  will  provide  instructors,  students,  and 
managers  great  flexibility  in  accessing  and  manipulating  training  software  and  data. 

One  of  the  most  difficult  problems  in  any  training  program  is  the  integration  of 
instruction  with  training  systems  responsive  to  the  changing  needs  of  the  student  The  hub 
proficiency  profile  system  will  contribute  much  to  overcoming  this  problem.  Currently, 
integration  of  instruction  with  training  systems  is  bypassed  by  requiring  all  trainees  to 
attend  the  same  courses  and  to  receive  about  the  same  content  and  exposure  to  activities. 
Research  and  experience  indicate  that  a  group  "lock-step"  method  does  not  optimize 
learning.  Lock-step,  in  fact,  may  result  in  students  who  perform  below  their  potential.  The 
problem  is  compounded  because  IPs  may  lack  sufficiently  detailed  student  data  to  optimize 
guidance  to  individuals.  Students'  scores  on  written  tests  and  IP  ratings  of  flying 
performance  currently  lack  the  detail  and  sensitivity  to  support  truly  individualized 
programs.  Moreover,  training  devices  cannot  be  adjusted  to  deliver  appropriately  detailed 
learning  opportunities  unless  individual  abilities  and  progress  can  be  monitored,  accurately 
assessed,  and  applied  to  each  training  event. 

One  of  the  most  salient  features  of  the  modeling  and  simulation  tools  being 
proposed  is  the  capability  to  measure  individual .  performance  across  all  phases  of  training. 

This  includes  academic  testing,  performance  checks  using  simulated  aviation  tasks  (at 
several  levels)  ,  and  performance  in  the  training  aircraft.  To  collect  and  manage  training 
data  of  this  magnitude,  a  central  database  system  is  essential.  It  must  contain  detailed 
records  of  student  performance  (in  the  form  of  proficiency  profiles).  Data  from  academics 
tests,  training  devices  and  simulators,  and  aircraft  sorties  should  be  used  in  the  profiles.  As 
a  student  progresses  through  the  program,  data  from  each  training  episode  will  be 
downloaded  to  the  hub  database  and  analyzed  to  update  the  respective  profile. 
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As  a  student  enters  SUPT,  each  individual’s  profile  will  include  demographics,  data 
from  prior  education  and  training,  and  other  relevant  aptitude  and  achievement  data.  As 
data  are  added  from  current  training,  the  profile  will  expand  to  yield  an  increasingly 
complete,  definitive  picture  of  student  performance.  Skill  areas  in  need  of  improvement 
will  be  highlighted  and  the  student  will  receive  appropriate  feedback  and  individual 
guidance:  Thus,  the  SUPT  hub  proficiency  profile  system  will  provide  a  dynamic,  highly 
individualized  tool  to  assist  students  toward  successful  completion  of  SUPT. 

The  SUPT  hub  will  be  accessed  by  students,  instructors,  and  managers  via  portable 
electronic  trainers  (PETs),  microcomputers  with  built-in  modem  capability,  linked  via  local 
area  networks  (LANs).  Each  student  will  use  a  PET  to  access  on-line  training, 
performance-support  software,  and  one’s  own  proficiency  profile  from  the  classroom, 
personal  quarters,  simulation  training  laboratory,  squadron-ready  facility,  or  even  the 
training  aircraft.  Figure  1  shows  the  communication  links  between  remote  sites  and  the 
SUPT  hub  within  the  training  base  LAN. 

Visual  Displays.  Visual  displays  are  extremely  important  to  pilot  training 
activities  because  of  the  need  to  integrate  cognitive  and  perceptual  skills  to  produce  high 
levels  of  performance  in  dynamic  flight  environments.  Three  types  of  display  technologies 
are  described  for  implementation  into  academics  and  simulator  training  phases  of  SUPT. 

Large-Screen  Displays.  Large-screen  displays  (LSDs)  are  electronic  devices  for 
projecting  computer-generated  imagery  and  digitized  photography.  A  classroom  instructor 
could  use  an  LSD  to  display  electronically  generated  word  slides,  graphs,  charts,  figures,  or 
maps,  in  the  same  way  as  blackboards  (whiteboards),  overheads,  and  film  projectors  are 
now  used.  A  unique  advantage  of  LSDs  over  older  media  is  that  they  support  active 
versus  passive  learning.  With  LSDs,  students  can  interact  directly  with  information  and 
see  changes  resulting  from  their  inputs,  an  obvious  advantage  to  the  learning  process. 
With  a  computer-image  generator  connected  to  the  LSD,  an  IP  could  demonstrate  cockpit 
and  out-the-window  visual  phenomena  similar  to  those  afforded  by  flight  simulators.  This 
would  enable  a  wide  variety  of  instructional  activities. 

Some  examples  may  help  to  illustrate  potential  interactive  applications  for  SUPT 
topics.  A  terrain  map  or  simulated  terrain  area  could  be  displayed  in  such  a  way  that 
students  could  "fly"  in  real  time  over  the  area  as  seen  from  the  cockpit.  They  could 
interact  with  information  by  changing  certain  flight  parameters,  or  selecting  or 
designating(identifying)  salient  ground  features  or  positions  through  joystick  inputs.  Such 
exercises  would  help  familiarize  students  with  local  flight  areas,  restrictions,  and 
procedures.  Basic  formation  flying  concepts  could  be  demonstrated  and  enable  instructors 
to  execute  control  inputs  using  a  joystick  or  mouse  to  show  various  viewpoints  of  pilots 
during  a  formation  flight.  Aircraft  position  could  be  changed  or  “flown”  in  real  time 
relative  to  other  aircraft  to  illustrate  envelopes  of  safe  flight,  closure  rates,  or  common 
errors.  Another  application  of  the  LSD  would  be  to  show  a  full  array  of  active  cockpit 
instruments  that  the  instructor  manipulates  during  simulated  flight  and  highlights  certain 
instruments  or  sequential  changes  to  demonstrate  correct  scanning  patterns,  cross-checks, 
instrument  procedures,  etc.  Students  could  use  the  same  medium  to  demonstrate  their 
knowledge  and  to  practice  identification  skills  following  instructors'  demonstrations. 
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Figure  1 

LAN  Network  Including  Hub  and  Remote  Training  Sites 


LSDs  have  gained  some  acceptance  in  the  last  few  years  in  academic  classrooms 
both  in  and  out  of  the  military  in  spite  of  high  costs.  However,  recent  advances  and  growth 
in  this  technology  have  reduced  the  cost  of  high  quality  LSDs.  The  Navy's  T-45  training 
program  already  uses  LSDs  in  its  classrooms.  Continuing  advances  in  flat  panel 
technology  are  expected  to  make  these  types  of  displays  increasingly  attractive  for 
classrooms. 

To  be  considered  an  LSD,  the  display  must  have  a  diagonal  dimension  of  four  feet 
or  more.  Rear-screen  projection  systems  enable  viewers  to  move  about  in  front  of  the 
display  without  disrupting  the  image  and  may  be  most  practical  for  the  classroom.  Image 
quality  is  normally  a  function  of  cost.  For  use  in  a  typical  SUPT  academic  environment 
where  viewers  may  sit  up  to  50  feet  from  the  screen,  resolution  and  brightness  should  be 
the  maximum  available. 

All  six  problems  areas  identified  in  this  research  effort  can  be  improved  with  LSD 
technology.  Visual  displays  in  an  academics  environment  are  envisioned  primarily  as 
teaching  tools  to  impart  knowledge  and  foundation  skills.  Thus,  the  best  apparent  SUPT 
application  is  using  LSDs  to  support  the  building  of  basic  skills  and  knowledge  as 
prerequisites  to  advanced  flying  skills. 

Remote  Three-Dimensional  Displays.  Adding  depth  cues  to  simulation  displays 
can  greatly  enhance  ability  to  perceive  distance  and  spatial  relationships.  Adding  a  third 
dimension  permits  the  student  to  engage  in  realistic  aircraft  control  practice.  Depth 
perception  is  a  function  of  several  different  types  of  visual  cues  and  dynamic  perceptual 
processes.  For  example,  the  brain  constantly  monitors  and  interprets  differences  in  image 
placement  perceived  by  each  eye.  Several  interface  technologies  use  different  approaches 
to  simulate  3D  vision  within  computer-generated  simulations.  Remote  3D  displays  project 
3D  imagery  from  flat  display  surfaces  that  are  viewed  from  a  distance  (e.g.,  computer  or 
video  monitor). 

Polarized  Systems  present  separate  images  to  the  right  and  left  eye  by  alternating 
horizontal  and  vertical  polarizations  of  the  image  at  a  very  fast  rate.  The  viewer  wears 
glasses  with  polarized  lenses.  This  gives  each  eye  a  slightly  different  view  of  the  scene 
and,  hence,  creates  the  perception  of  three  dimensions. 

Shutter  Systems  employ  special  eyeglasses  with  shutters  to  control  the  display  of  a 
slightly  different  perspective  to  each  eye.  The  function  of  the  shutters  is  to  alternate  the 
opaqueing  (blocking)  of  information  to  each  eye.  The  scene  alternates  between  a  slightly 
right  offset  and  a  slightly  left  offset  perspective  at  a  very  fast  rate  which  the  brain  perceives 
as  a  continuous  3D  image. 

Autostereoscopic  Displays  use  a  high-quality  video  raster  scan  color  screen 
combined  with  special  eyewear.  To  generate  3D  visual  imagery,  the  screen  displays  the 
left  and  right  halves  of  the  scene  on  alternating  columns  of  video  pixels  at  a  rate  of  30 
frames  per  second.  The  left  image  appears  in  the  odd  columns  and  the  right  image  appears 
on  the  even  columns.  The  user,  by  remaining  in  a  specified  viewing  position,  receives 
independent  sets  of  information  to  each  eye,  and  thus  perceives  a  continuous  3D  image. 
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The  visual  capabilities  of  remote  3D  displays  may  be  useful  in  a  variety  of  training 
phases  where  it  is  important  for  students  to  gain  an  understanding  of  3D  flight  dynamics 
(e.g.,  interactions  encountered  among  flight  parameters  such  as  altitude,  closing  range, 
attitude,  yaw,  pitch,  role,  etc.)  or  practice  within  an  environment  where  3D  cues  are 
essential  to  the  task  at  hand. 

The  Display  for  Advanced  Research  and  Training  (DART).  DART  simulation 
technology  provides  out-the-window  visual  imagery  for  the  flight  simulation  environment. 
It  is  designed  to  complement  a  variety  of  simulation  cockpit  configurations.  In  the  present 
case,  it  is  recommended  for  use  with  the  unit  training  device  (UTD)  flight  simulator. 
DART  was  developed  to  provide  high  quality  visual  display  characteristics  at  very  low 
cost.  In  tandem  with  image  generator  technology  of  comparable  value,  DART  furnishes 
sufficient  visual  quality  to  provide  useful  training  for  aircraft  such  as  the  F- 16. 

One  of  the  geometric  display  configurations  for  the  DART  is  a  dodecahedron 
composed  of  eight  interlocking  display  channels.  Barco  data  rear  screen-mounted 
projectors  (one  per  channel)  for  the  DART  exceed  1,000  lines  of  video  resolution.  Display 
surfaces  are  located  about  one  meter  from  the  pilot's  eyepoint.  Resolution  to  the  pilot's  eye 
is  4.75  arcminutes/pixel.  Field  of  regard  is  about  300  degrees  horizontally  and  150  degrees 
vertically.  A  Polhemus  magnetic  tracker  is  used  to  track  the  head  movement,  and  data  are 
used  to  reduce  image  generator  (IG)  channel  processing  requirements.  This  is 
accomplished  by  dimming  the  appropriate  projectors  for  unused  imagery  behind  the  pilot's 
head.  Overall  brightness  (7  footLamberts)  and  contrast  characteristics  (30:1)  are  excellent 
in  comparison  to  previous  simulator  display  designs  and  overall  system  cost.  The  overall 
result  is  true  "wraparound"  visual  imagery  in  full  color  with  high  visual  fidelity.  The  Mini- 
DART  uses  fewer  IG  channels  and  display  screens  to  accomplish  almost  the  same 
capability  as  the  full-size  DART  but  occupies  a  much  smaller  space,  has  greater  portability, 
and  is  more  economical. 

Immediate  training  effects  anticipated  from  employment  of  the  DART  can  improve 
transfer  of  training  from  simulator  to  aircraft.  Long-term  effects  can  be  estimated  based  on 
a  substantially  expanded  role  of  simulation  training  in  SUPT. 

DART  technology  is  estimated  to  have  excellent  evolutionary  potential.  Video 
projection  technology  used  in  DART  is  advancing  rapidly  and  the  industry  forecast  is  for 
stable  long-term  growth.  This  suggests  commercial,  off-the  shelf  hardware  will  be  readily 
available  for  system  updates  during  the  projected  life  cycle  of  the  program. 

For  training  the  overhead  traffic  pattern,  DART  would  offer  the  first  opportunity 
for  SUPT  students  to  visually  experience  the  entire  maneuver  in  real  time  in  a  flight 
simulator.  The  DART  provides  students  with  a  visual  environment  in  which  they  can 
develop  recognition  skills  for  key  visual  cues  and  integrate  cues  with  crucial  flying 
procedures.  The  information  processing  demands  which  create  much  of  the  difficulty  in 
the  landing  pattern  could  be  better  managed  if  the  trainee  had  sufficient  practice  time  in  a 
high  quality  simulator  equipped  with  the  DART. 

Head-Mounted  Displays  (HMDs).  Basically,  an  HMD  is  an  individual  headgear 
with  close-fitting  lenses.  Typically,  the  HMD  is  connected  to  a  computer  that  generates 
visual  images  on  two  small  screens  (one-and-a-half  to  two-inches  diagonal)  positioned  in 


17 


front  of  each  eye.  With  the  help  of  magnifying  power,  the  lenses  can  encompass  the  entire 
normal  field  of  human  vision.  Since  the  viewer  is  visually  “partitioned”  from  the  actual 
surrounding  environment  and  "immersed"  within  the  simulated  scene,  the  effect  on  the 
viewer  is  one  of  experiencing  a  “virtual  reality.” 

There  are  several  technical  approaches  or  variations  of  HMD  design.  Each 
significantly  modifies  the  overall  perceptual  effects.  Each  involves  functional  components 
for  image  position-tracking,  image  opaqueing,  and  degree  of  binocular  or  monocular 
characteristics  of  the  imagery  as  perceived  by  the  viewer. 

Position  tracking  allows  the  viewer  to  move  their  line  of  site  and  head  position, 
while  the  simulated  imagery  changes  to  accommodate  the  shift  in  the  viewer's  perspective. 

For  example,  as  one  moves  closer  to  a  simulated  aircraft,  it  becomes  larger,  and 
perspective  angles  change  realistically.  As  the  viewer’s  head  turns,  the  available  field  of 
view  shifts  accordingly.  HMDs  can  also  be  designed  so  that  the  computer  tracks  eye 
position  and  changes  the  simulated  environment  relative  to  the  viewer's  eye  movements. 
Position  trackers  use  mechanical,  optical,  or  electromagnetic  devices  to  sense  changes  in 
the  viewers  position  and  line  of  sight. 

Opaque  HMDs  permit  viewers  to  see  only  simulated  imagery,  thereby  “opaqueing” 
or  blocking  out  the  viewer's  actual  visual  environment.  See-through  HMDs,  by  contrast, 
enable  the  user  to  see  the  surrounding  physical  environment  along  with  the  simulation  by 
superimposing  simulated  imagery  on  top  of  the  display  lens.  Both  methods  have 
advantages  or  disadvantages  based  upon  effects  desired. 

Some  HMDs  provide  binocular  depth  cues  (shift  in  perspectives  perceived  by  the 
right  and  left  eye)  using  similar  techniques  featured  by  remote  3D  displays  (described 
above).  Binocular  HMDs  have  advantages  and  limitations.  Perception  of  depth  is  most 
pronounced  within  relatively  short  distances,  up  to  about  30  feet.  Beyond  that,  binocular 
disparity  is  less  effective  as  a  distance  cue.  For  this  reason,  binocular  HMDs  would  be 
most  useful  for  aircrew  tasks  performed  fairly  near  the  pilot. 

Monocular  HMDs,  on  the  other  hand,  can  be  used  for  simulating  imagery  for  tasks 
which  do  not  require  critical  depth  cues.  In  addition,  monocular  HMDs  require  less 
computing  power  since  the  computer  needs  to  create  only  one  visual  scene  instead  of  two. 
In  many  simulator  applications,  lesser  demand  for  computer  capacity  can  significantly 
reduce  system  cost.  Also,  various  HMD  designs  differ  in  size,  weight,  image  resolution, 
image  brightness  and  contrast,  and  color  capability.  Some  low  fidelity  devices  that  are 
currently  under  development  may  provide  visual  immersion  within  simulated  imagery  with 
the  viewer  having  to  wear  no  more  than  a  pair  of  lightweight  visors  and  a  lightweight 
tracking  sensor. 

HMD  technology  may  enhance  the  effectiveness  of  training  for  the  overhead  traffic 
pattern.  In  the  academic  phase,  HMDs  could  be  used  to  accelerate  the  student’s 
familiarization  with  visual  information  necessary  for  performing  this  difficult  maneuver. 
A  virtual  representation  of  the  landing  environment  would  allow  a  novice  to  experience 
spatial  and  geometric  aspects  of  the  landing  area  before  ever  entering  the  aircraft.  Students 
could  study  flight  path  envelops,  ground  references,  and  visual  cues  from  a  variety  of 
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eyepoint  perspectives.  Such  information  would  be  valuable  in  helping  students  develop 
accurate  mental  models  of  flying  tasks. 

The  major  impact  of  HMD  technology  may  by  in  training  formation  flight.  To 
maximize  training  benefits  in  the  aircraft,  ground  training  in  a  simulator  with  a  state-of- 
the-art  HMD  should  precede  formation  flight  in  the  aircraft.  Field  of  view  and  display 
resolution  will  be  particularly  important  parameters  because  of  the  critical  nature  of  visual 
cueing  associated  with  formation  flight  training.  A  wider  field  of  view  and  better 
resolution,  than  is  currently  featured  by  most  HMDs,  is  needed  to  allow  the  pilot  sufficient 
flexibility  to  view  other  aircraft  during  formation  flight.  The  pilot  must  be  able  to  resolve 
surface  details  of  other  aircraft  during  formation  because  these  cues  allow  precise 
judgments  concerning  distance  and  closure  rates.  Thus,  some  additional  improvements  are 
still  needed  in  HMD  capabilities  before  they  will  be  100%  effective  for  training  detailed 
and  close-up  visual  tasks. 

Figure  2  shows  an  artist's  conception  of  the  UTD  equipped  with  each  of  the  two 
proposed  visual  display  systems,  the  Mini-DART  on  the  left  and  a  lightweight  visor  on  the 
right. 

Upgrades  in  Computer- Assisted  Instruction  and  Portable  Electronic  Trainers. 

Recent  developments  in  microprocessors,  application  software,  computer  architecture, 
communications  hardware  and  software,  and  multimedia  technology  have  resulted  in  tools 
that  offer  great  potential  for  improving  the  quality  and  capability  of  computer-aided 
instruction  (CAI)  curriculum  and  training  activities.  Periodic  assessment  of  knowledge 
acquisition,  performance  analysis,  and  the  delivery  of  appropriate  remedial  training  can 
now  be  more  easily  designed  into  instructional  software.  Systems  containing  such  features 
can  be  networked  in  order  to  (a)  expand  economy  of  scale  and  program  flexibility,  (b) 
enhance  training  synergy  by  permitting  student  interaction' with  subject  matter,  instructors, 
and  other  students  in  the  classroom  and  at  remote  locations,  and  (c)  allow  individual 
students  freedom  and  flexibility  to  access  varieties  of  information  and  instructional 
programs  on  a  24-hour  basis.  Advancements  in  microprocessor  technology  have  resulted 
in  the  development  of  powerful,  lightweight,  and  compact  systems  capable  of  providing  a 
wide  range  of  training  and  management  functions. 

The  ongoing  revolution  in  computer  technology  promises  reductions  in  cost  and 
size  of  hardware  systems  and  dramatic  increases  in  data-storage  capacity  and  functional 
capabilities.  Portable  systems  can  operate  on  battery  power  and  be  designed  to  dock  with 
(plug  into)  full-size,  desktop  training  stations.  It  is  hard  to  predict  specific  developments  in 
applications  during  the  next  20  years  from  a  SUPT  perspective.  However,  it  is  likely  that 
instructors  and  students  would  benefit  from  a  microcomputer-based  PET  that  would 
provide  each  individual  with  advanced  training  support  and  communications  capabilities  at 
any  location  on  the  training  base.  The  PET  would  be  small  enough  to  fit  into  a  flight  suit 
pocket,  yet  would  provide  students  with  immediate  access  to  their  individual  student 
profile  and  communication  with  any  instructor  pilot.  It  would  support  a  variety  of  training, 
time-management,  and  scheduling  functions  to  increase  the  efficiency  of  individual  and 
team  training  activities. 
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Figure  2 

Types  of  Visual  Displays 


The  PET  would  be  combined  with  desktop  systems  to  provide  all  the  computer  and 
simulation  capabilities  needed  for  academics.  Desktop  systems  can  be  equipped  with 
special  interface  devices  to  accommodate  built-in,  quadruple-speed,  compact  disc,  read¬ 
only  memory  (CD-ROM)  drives;  modems  for  high-speed  communications  and  data 
transfer;  high-resolution  displays  for  both  flat-display  and  3D  simulations;  and  stereo  or 
even  3D  (localized)  sound.  Such  systems  would  support  a  broad  range  of  training 
activities,  including  the  development  of  high-performance  cognitive  skills  needed  to 
maintain  situational  awareness  in  the  cockpit  while  executing  time-critical  tasks.  Such 
training  would  provide  a  stronger  foundation  of  skills  prior  to  a  student's  engagement  in 
full-fidelity  flight  simulation  training.  Intermediate  level  work  stations  with  some 
advanced  microcomputer  features  could  be  installed  in  classrooms  and  even  students' 
private  quarters  at  a  reasonable  cost,  while  specialized  displays  and  simulation  capabilities 
would  be  reserved  for  the  simulation  training  laboratory  (discussed  below). 

The  capabilities  of  new  microcomputer  systems  and  software  would  enable  learners 
to  play  a  more  active  (interactive)  role  in  learning  and  advancing  through  training  activities 
as  opposed  to  traditional  group  lecture/demonstrations  and  passive  media  presentations  in 
the  classroom  or  media  lab.  The  location  of  learning  activities  would  become  more 
flexible  because  of  multiple  training-delivery  modes  and  options. 

Linking  microcomputers  together  via  a  LAN  system  has  many  practical  advantages. 
A  LAN  can  be  configured  in  a  variety  of  ways  to  enable  students  to  access  a  full  range  of 
desktop  applications  as  well  as  real-time  communications  and  interactive  training 
programs.  Such  functions  would  include  performance-support  programs  like 
wordprocessors,  spreadsheets,  and  databases  as  well  as  specialized  pilot-training 
courseware.  From  a  more  expansive  perspective,  students  could  use  such  desktop  systems 
to  gain  immediate  access  to  military,  private,  and  commercial  communication  lines  to  and 
from  literally  anywhere  in  the  world  (e.g.,  Internet  and  worldwide  web).  Instructors  could 
transmit  to  and  receive  from  students,  staff,  and  command  personnel  various  types  of 
verbal  and  graphic  communications.  A  microcomputer  LAN  can  also  support  multiple 
security  levels  which  restrict  or  facilitate  access  to  designated  materials  or  data  to  specific 
personnel  (e.g.,  grade  records).  Instructors  could  make  changes  to  courseware  so  students 
always  receive  up-to-date  materials.  Scheduling  of  events  (e.g.,  training  flights)  could  be 
input  on  the  network  so  every  user  has  immediate  information  updates.  Special  integration 
with  other  supercomputer  systems  can  also  provide  desktop  users  with  special  capabilities 
that  are  not  possible  with  stand-alone  systems. 

The  PET  and  associated  training  software  could  be  applied  to  all  six  problem  areas 
identified  in  this  research  study.  The  basic  improvements  to  current  training  would  be 
individualized  training,  performance  tracking,  automated  performance  assessment,  and  an 
extension  of  the  current  academic  training  capabilities.  An  artist's  conception  of  the  PET 
and  desktop  training  station  is  illustrated  in  Figure  3. 

Instructional  Simulations.  More  powerful  microcomputer  systems  have  also 
made  it  possible  to  combine  simulation  with  CAL  This  becomes  a  very  important 
capability  when  dealing  with  learning  of  a  dynamic  (versus  a  static)  nature.  Instructional 
simulations  can  be  used  to  teach  complex  skills  that  consist  of  intellectual,  perceptual,  and 
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Figure  3 

Artist's  Conception  of  PET  and  Desktop  Training  Station 
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motor  ability  components.  The  design  of  instructional  simulations  differs  from 
conventional  CAI  whereby  learners  typically  receive  sequences  of  text  and  graphic 
information  that  tests  students'  knowledge  with  various  questions.  Such  learning 
sequences  are  essentially  static  and  are  based  on  the  recall  of  facts,  identification  of 
concepts,  executing  recall  of  procedural  steps,  etc.  In  contrast,  instructional  simulations  go 
a  significant  step  further  to  engage  learners  with  dynamic  tasks  that  integrate  knowledge  of 
the  task  with  the  complex  cognitive  challenges  associated  with  fast-changing  environments 
and  decision  structures.  For  SUPT  applications,  instructional  simulation  would  extend  the 
value  of  desktop  trainers.  Efficiency  in  the  overall  program  would  improve  because 
students  would  develop  cognitive  skills  that  readily  transfer  to  advanced  phases  of 
simulator  and  flight  training. 

Instructional  simulations  replicate  the  dynamics  of  task  environments  using  visual 
and  sometimes  audio  media.  Learners  interact  with  dynamic  representations  of  either 
physical  or  abstract  environments.  They  can  learn  both  abstract  aerodynamic  principles, 
which  cannot  be  demonstrated  using  common  instructional  tools  and  media,  or  students 
can  build  skills  practicing  hands-on  tasks  that  require  great  speed  and  accuracy.  During  the 
simulated  tasks,  the  computer  monitors  and  records  student  responses,  provides  visual  or 
acoustic  cues,  performance  feedback,  and  advice  on  increasing  level  of  skill.  Instructional 
simulations  can  also  automatically  alter  task  environments  according  to  learner 
performance. 

Complex  flying  tasks  usually  involve  the  execution  of  multiple  subtasks  (parts  of 
the  whole  task)  to  accomplish  a  single  goal.  Part-task  instructional  simulation  is  one 
method  for  reducing  difficulties  learners  have  in  understanding  and  performing  these  tasks. 

Microcomputer  training  is  a  platform  for  implementing  part-task  training  for  a  variety  of 
aviation  tasks  that  are  primarily  dependent  on  intellectual  skills  rather  than  motor  abilities. 

Part-task  instructional  simulation  enables  learners  to  move  smoothly  from  a  state  of  no 
knowledge  of  a  task  to  a  basic  understanding  and,  eventually,  to  high  proficiency  on  one  or 
more  complex  task  components. 

The  following  are  examples  of  potential  SUPT  instructional  simulations: 

1.  Instrument  Interpretation  Training— Desktop  simulation  training  that  would 
provide  real-time  feedback  to  students  on  their  ability  to  monitor  and  interpret  dynamic 
situations  with  navigation  instruments  such  as  NAVAID  displays. 

2.  Concepts  of  Radial  Geometry  Training— Simulations,  animations,  and  video 
clips  would  be  integrated  into  a  series  of  exercises  to  teach  the  geometric  nature  of 
navigation.  It  would  go  beyond  interpreting  instrument  displays  by  permitting  students  to 
manipulate  concept  models  which  underlie  navigation  problems. 

3.  “What  If’  Training— Instructional  simulations  would  allow  a  student  to 
manipulate  graphic  representations  of  aircraft,  navigation  instruments,  and  navigation  aids 
(NAVAIDS)  to  discover  strategies  for  interpreting  and  understanding  the  instruments.  For 
example,  changing  the  position  of  a  graphic  representation  of  the  aircraft  would  update 
settings  on  relevant  navigation  instruments  respectively.  Such  real-time  animated  training 
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scenarios  would  accelerate  the  students'  grasp  of  many  types  of  complex  phenomena  and 
tasks. 

4.  Fix-To-Fix  Instructional  Simulation-Instead  of  attempting  to  visualize  dynamic 
situations  by  chair  flying,  students  would  be  able  to  set  various  factors  such  as  wind, 
airspeed,  and  fuming  radius  on  visually  simulated  instruments  and  flight  map  and  observe 
results  in  real  time.  The  simulation  would  include  such  factors  as  radio  communications 
with  the  tower  during  maneuvers. 

Instructional  simulations  could  be  applied  to  all  six  problem  areas  with  special 
emphasis  on  dynamic,  cognitive  skills  that  can  be  trained  without  the  use  of  a  full-fidelity 
flight  simulator. 

Simulation  Training  l  aboratory— The  simulation  training  laboratory  is  proposed, 
in  this  report,  as  an  additional  component  to  the  SUPT  syllabus.  The  creation  of  this 
learning  environment  is  possible  through  a  combination  of  technologies  which  we  describe 
in  this  report.  Functionally,  the  training  laboratory  would  bridge  what  amounts  to  a 
training  gap  in  the  syllabus  between  academics  and  simulator/aircraft  training  phases. 
Presently,  the  student  inculcates  knowledge  from  lectures,  printed  material,  and  CAI 
(academics),  then  attempts  to  apply  that  knowledge  to  develop  flying  skills  in  the  simulator 
and  aircraft.  Most  students  encounter  some  degree  of  difficulty  with  this  transition.  An 
environment  for  developing  basic  hands-on  piloting  skills  is  needed,  and  new  technology 
can  provide  it  at  reasonable  cost.  Research  also  shows  that  student  pilots  benefit  from, 
indeed  prefer,  opportunities  to  experiment,  discover,  and  practice  without  instructors. 
Also,  the  synergism  of  two  or  more  students  learning  jointly  can  improve  motivation, 
learning,  and  retention,  and  also  engender  a  favorable  climate  for  team  training  and  crew 
cooperation. 

The  specific  objective  of  the  simulation  training  laboratory  would  be  to  produce 
highly  transferable  skills  for  subsequent  simulator  and  aircraft  training.  The  training 
laboratory  would  provide  entry-level  simulation  experiences  well  beyond  what  can  be 
produced  using  conventional  CAI.  It  would  also  obviate  the  need  to  resort  to  chair  flying 
(abstract  mental  rehearsal).  The  simulation  training  lab  would  provide  a  vehicle  for 
exploiting  some  low-cost  simulation  technologies  to  better  integrate  academics  with 
simulator/aircraft  training  activities. 

The  simulation  training  lab  would  feature  microcomputer-based,  part-task  training 
systems  and  PET  training  stations  (described  above).  A  selective-fidelity  approach  would 
be  used  in  the  design  of  part-task  trainers  to  match  the  respective  training  task.  For 
example,  a  trainer  for  practicing  basic  aircraft  control  skills  would  feature  only  stick-and- 
throttle  controls  with  force-feedback  (control  loaded)  response,  ADI  (attitude  indicator), 
VVI  (vertical  velocity  indicator),  HSI  (horizontal  situation  indicator),  and  ALT  (altimeter). 

A  navigation  procedures  trainer  would  consist  of  a  microcomputer-based  system  that 
simulates  navigation  instruments  and  controls  on  a  touchscreen  and  provides  a  large 
display  for  digitized  maps  and  flight  planning  information.  The  cockpit  would  interface 
with  the  SUPT  hub  computer  system  to  download  various  preprogrammed,  computer¬ 
generated  scenarios  depending  upon  the  student's  choices,  current  training  status,  and  level 
of  achievement  on  cockpit  procedures  training  as  indicated  by  his/her  proficiency  profile. 
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The  training  laboratory  would  be  available  to  students  during  all  phases  of  the 
training  cycle,  and  the  hours  of  operation  would  permit  maximum  accommodation  of 
student  schedules.  Figure  4  shows  an  artist's  conception  of  two  selective  part-task  trainers, 
one  for  practicing  basic  aircraft  control  skills  on  the  left  and  the  other  for  practicing 
navigation  procedures  on  the  right. 

Unit  Training  Device  (TJTDV- Aircraft  time  allocated  to  each  SUPT  student  is 
limited.  Advanced  flight  simulation  technology  could  substantially  address  this  challenge 
by  producing  more  capable  student  pilots  on  entering  flying  training  phases.  The 
assumption  is  that  students  could  perfect  many  skills  prior  to  training  in  the  aircraft  given 
sufficiently  capable  simulators. 

The  UTD  will  replicate  the  functional  capabilities  of  the  JPATS/-T-38. 
Currently,  the  Air  Force  is  procuring  UTDs  for  F- 16  training  and  is  planning  to  procure 
UTDs  for  the  T-38  training  aircraft.  The  UTD  approach  uses  engineering  and  software 
design  methods  developed  by  the  Aircrew  Training  Research  Division  of  Armstrong 
Laboratory  in  its  Multitask  Trainer  (MTT)  program. 

All  instrumentation,  flight  controls,  and  panels  will  be  accurately  positioned.  All 
aircraft  systems  will  be  functional.  Simulation  fidelity  will  be  sufficient  to  perform  all 
required  syllabus  events  associated  with  the  JPATS  and  T-38  training  missions.  A  motion 
platform  for  the  UTD  is  not  required. 

Since  the  greatest  expense  in  the  development  of  any  flight  simulator  is  the 
simulation  software,  existing  T-38  OFT  software  should  be  converted  to  operate  with 
microcomputers  selected  for  the  UTD.  This  use  of  existing  government  software  can 
greatly  reduce  the  development  time  and  costs.  Aircraft  Line-Replacement  Units  (LRUs) 
should  be  simulated  in  software  instead  of  using  the  actual  aircraft  LRUs  for  high  fidelity 
simulation.  UTD  computer  simulation  software  must  be  bit  level-compatible  with  the  T- 
38  OFT  simulation  software. 

The  UTD  will  use  commercial,  off-the-shelf,  simulator  components  instead  of 
actual  aircraft  instrumentation  and  controls.  This,  combined  with  use  of  microcomputers, 
will  greatly  reduce  the  cost  and  overall  size  of  the  UTD.  It  will  also  preclude  the  need  for 
special  simulator  facilities  to  house  the  device.  The  UTD  will  have  on-board  air 
conditioning  for  operation  in  normal  office  environments  using  standard,  110  volt,  20  amp 
electrical  power. 

The  UTD  will  be  self-contained,  excluding  the  instructor-operator  station  (IOS), 
image  generator  and  the  visual  display  system.  The  simulation  computers,  cockpit 
interfaces,  power  supplies,  control  loading  for  the  flight  control  stick,  and  air  conditioner, 
can  be  housed  within  the  UTD  cabinet.  The  UTD  will  be  easily  operated  and  can  provide 
the  flexibility  to  meet  the  training  needs  of  the  JPATS  and  T-38  pilots.  Students  will  be 
able  to  operate  the  UTD  independently  or  with  an  instructor  depending  upon  the  training 
requirements.  The  UTD  will  be  easily  transportable  by  truck  or  aircraft  and  will  have 
wheels  to  facilitate  movement.  The  average  set-up  time  for  the  device,  from  crate  to 
operation,  will  be  approximately  one  hour.  The  footprint  of  the  UTD  will  be  less  than  10 
feet  x  10  feet.  UTD  software  will  easily  integrate  with  commercially  available  visual  or 
digital  radar  landmass  systems,  image  generators,  and  various  types  of  visual  displays. 
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Figure  4 

Artist's  Conception  of  Selective  Part-Task  Trainers 
for  Aircraft  Control  and  Navigation  Procedures  Practice 


The  UTD  will  be  employed  at  advanced  stages  of  pilot  training,  prior  to  and  integrated 
within,  flying  training  in  the  aircraft. 

Simulation  Networking  Technologies— As  an  R&D  agency  with  considerable 
experience  in  distributed  interactive  simulation  (DIS),  AL/HRA  is  in  a  favorable  position 
to  recommend  and/or  design  multiship  network  simulations  for  SUPT  training 
applications.  The  technical  risk  of  developing  and  implementing  such  systems  is  relatively 
low  because  SUPT  technical  requirements  are  now  well  within  the  current  DIS  capability. 

Two-  or  four-ship  combinations  of  simulator  cockpits  housed  in  the  same  training 
squadron  (T-37  or  T-38  for  first-phase  applications)  would  employ  LAN  technologies  in 
combination  with  low-cost  simulation  technologies  previously  developed  and 
demonstrated  in  MTT  R&D  by  AL/HRA.  Multiship  simulation  would  provide  the  SUPT 
program  with  the  capabilities  of  having  (at  a  minimum)  two  pilots  (such  as  instructor 
trainee  in  separate  aircraft,  or  lead/wingman)  fly  a  full  range  of  two-ship  missions  or 
maneuvers  as  specified  by  the  training  syllabus.  Each  simulator  would  provide  highly 
realistic  representations  of  the  flight  environment  including  other  aircraft  in  the  scenario. 
High  fidelity  simulation  models  of  aerodynamics,  engine,  radio,  and  all  associated  avionics 
functions  and  subsystems  would  be  provided.  The  networked  simulators  would  be 
equipped  with  either  DART  or  HMD  technology.  Flight  environments  modeled  in  a 
computer-generated  visual  database  would  include  local  areas,  practice  ranges,  and  other 
required  geographic  areas. 

Networking  Applications  and  Benefits.  Simulation  networking  for  SUPT  would  be 
superior  to  existing  technology  simply  because  these  capabilities  are  now  unavailable  in 
the  program.  Immediate  benefits  would  come  in  formation  flight  training  where  the 
advantages  of  interaction  between  pilots  in  separate  simulated  aircraft  would  become 
obvious.  Such  an  addition  to  the  program  can  be  expected  to  produce  substantial  gains  in 
student  pilot  performance  due  to  the  transfer  of  superior  ground  training  to  aircraft 
formation  flight  sorties. 

Specific  types  of  training  supportable  through  use  of  multiship  simulator 
networking  are  as  follows: 

1.  Formation  Flight-Skill  development  for  between-aircraft  closure  rates  and 
control  techniques,  aspect  angle,  heading  crossing  angle,  and  co-altitude  rejoins. 

2.  Overhead  Traffic  Pattern— One  aircraft  in  overhead  pattern,  multiple  aircraft 
in  pattern,  emergency  aircraft  recovery,  and  confused  tower  scenarios. 

3.  Multiship  Mission  Rehearsal— Exposure  and  rehearsal  of  complete  missions 
prior  to  actual  aircraft  sortie. 

Enhancement  of  formation  flight  training  for  all  aircraft  would  be  feasible,  but  cost 
effectiveness  determinations  will  require  additional  data  collection  beyond  the  scope  of  this 
study.  Optimal  phasing  of  technology  acquisition  would  be  based  upon  whether  simulator 
networking  would  be  more  beneficial  if  introduced  first  in  the  T-37  or  in  the  T-38.  As 
multiship  networking  costs  recede,  all  aircraft  training  would  logically  benefit  from 
networked  simulator  capabilities.  By  extension,  the  linking  of  SUPT  training  squadrons  at 
various  bases  could  be  considered.  Such  capabilities  obviously  transcend  current  SUPT 
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training  requirements.  However,  this  may  be  due  to  the  fact  that  DIS  technology  has  not 
been  available  until  recently. 

Networked  simulators  afford  the  AETC  community  the  means  to  explore  new 
horizons  in  advancing  the  training  for  USAF  pilots.  DIS  might  be  particularly  beneficial  in 
SUPT  depending  upon  the  degree  to  which  multiship  training  is  .expanded.  For  example, 
it  appears  quite  certain  that  such  capabilities  would  increase  SUPT  trainee  proficiencies  for 
introductory  basic  fighter  maneuvering  (BFM).  Beyond  that,  multiship  training  could  be 
expanded  to  encompass  considerably  advanced  levels  of  sophistication  of  training  to  better 
prepare  SUPT  students  for  selected  CCTS  assignments. 

Portable  Electronic  Trainer  fPETI  in  the  Cockpit.  Ever  since  flight  instructors 
began  training  students,  they  have  faced  the  challenge  of  recording  information  on  the 
student's  inflight  performance  that  is  useful  for  the  debrief.  IPs  have  traditionally  used 
notebook-style  checklists  and  rating  forms  on  a  "kneeboard"~a  small  notebook  that  is 
strapped  around  the  pilot's  leg  during  the  flight.  Kneeboards  have  drawbacks  as  training 
aids  because  (a)  there  is  little  space  for  written  information;  (b)  the  IP  must  look  down  to 
write  and  may  miss  important  student  behavior;  (c)  notes  must  be  manually  transferred  to 
some  type  of  filing  or  storage  system  for  use  beyond  the  immediate  debrief;  and  (d)  using 
kneeboards  during  night  sorties  is  awkward. 

All  of  these  drawbacks  could  be  corrected  by  modernizing  the  kneeboard  concept. 
The  PET  could  conceivably  be  equipped  with  a  special  set  of  functions  that  would  enable 
instructors  and  student  pilots  to  enter  relevant  flight  data  via  a  simplified  keyboard  or 
touchscreen  interface  with  minimal  distraction  during  flight.  A  high  resolution  screen 
(about  5  inches  x  5  inches)  would  display  a  simple  menu  that  could  be  illuminated  during 
night  flights.  Performance  notations  and  ratings  could  be  made  with  a  few  keystrokes  or 
screen  touches.  The  resulting  PET  data  would  interface  directly  with  the  Aeronautical 
Training  Recorder  (ATR,  described  below)  to  integrate  performance  ratings  and  notes  with 
respective  flight  events.  All  such  data  could  be  immediately  downloaded  to  the  SUPT  hub 
computer  after  the  flight  to  update  the  student's  proficiency  profile  and  to  use  for 
debriefing.  Whether  the  device  is  used  on  the  knee  or  attached  to  a  panel  in  the  cockpit,  it 
would  be  easily  attached  and  removed  and  designed  to  comply  with  flight  safety 
regulations. 

The  PET  could  also  be  used  by  IPs  during  flight  simulator  training  for  the  same 
performance-monitoring  purposes.  Increasing  the  quality  and  quantity  of  useful 
performance  data  available  to  IPs  during  debrief  was  emphasized  as  a  main  concern  of 
instructor  pilots.  IPs  must  work  with  a  number  of  students,  and  they  are  currently  unable 
to  remember  and/or  write  down  enough  information  to  optimize  student  guidance  during 
flight  training.  This  application  of  the  PET  would  extend  its  usefulness  to  all  phases  of 
training  from  academics  to  flight  training  in  the  aircraft.  PETs  would  help  keep  students 
on  track  by  keeping  them  informed  on  their  flying  progress  and  would  reduce  IP's 
workload  by  facilitating  student  profile  monitoring  and  inflight  performance  assessment 
tasks. 
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Aeronautical  Training  Recorder  (ATR).  The  ATR  would  capture  a  continuous 
sequence  of  real-time,  dynamic  flight  data  from  the  aircraft  during  training  sorties. 
Aircraft  data  would  be  integrated  with  local  terrain  database  models  for  purposes  of  re¬ 
creating  actual  training  flights  via  simulation.  This  capability  should  vastly  improve 
debriefing  capabilities. 

The  ATR  would  consist  of  three  major  components:  (a)  an  aircraft-mounted  (on¬ 
board)  microcomputer  with  high-capacity  data  storage;  (b)  an  on-board  Global  Positioning 
System  (GPS)  linked  to  the  ATR  computer;  and  (c)  a  ground-based  computer  system  that 
would  combine  aircraft  flight  data  (instrumentation  and  GPS)  with  a  local  terrain  database. 
The  on-board  computer  would  sample  instruments,  cockpit  acoustics,  and  radio 
communications  at  about  60  Hz  to  reproduce  the  flight  in  a  high-fidelity  simulation  format. 
The  recording  process  could  be  manually  engaged  and  disengaged  at  any  time  during  the 
flight  by  the  instructor  using  the  PET  as  the  inflight  ATR  interface. 

A  removable  data  packet  would  permit  easy  transfer  of  flight  data  to  ground-based 
systems.  The  ground-based  image  generator  would  reproduce  any  recorded  portion  of  the 
flight  from  a  variety  of  viewpoints  (tower  view,  inside-cockpit  view,  off-the-wing  view, 
stem  view,  etc.).  The  ATR  would  remove  the  IP's  challenge  for  remembering  dynamic 
details  about  each  training  sortie  to  effectively  debrief  the  student.  A  similar  technology  is 
currently  used  in  the  Air  Combat  Maneuvering  Instrumentation  (ACMI)  program. 
However,  the  SUPT  version  would  use  modernized  technology  at  a  much  reduced  cost. 

The  GPS  would  triangulate  time/position  data  of  the  aircraft  (at  a  suitable  level  of 
accuracy)  by  continuously  interrogating  navigation  satellites  and  synchronizing  the  aircraft 
position  data  (longitude,  latitude,  and  altitude)  with  the  instrumentation  data.  The  GPS 
system  would  require  some  aircraft  modifications  to  accommodate  receiving/transmitting 
antennae. 

The  ground-based  computer  system  would  translate  the  inflight  ATR  data  to  a 
format  which  would  enable  the  flight  to  be  simulated  and  replayed  in  a  high-fidelity  format 
on  the  UTD  or  in  a  medium-fidelity  format  on  classroom  training  stations. 

The  ATR  would  be  employed  at  advanced  stages  of  pilot  training  when  the  student 
pilot  begins  flying  sorties  in  the  aircraft.  The  ATR  simulations  would  be  used  to  debrief 
student  pilots  in  a  passive  (view  only)  format  so  their  performance  could  be  reviewed  and 
discussed  with  the  IP.  ATR-captured  training  sorties  could  also  be  employed  to  enable 
students  to  re-experience  a  sortie.  By  downloading  ATR  data  to  the  UTD,  a  student  could 
re-fly  exactly  the  same  maneuvers  in  the  UTD  that  were  found  to  be  most  difficult  in  the 
aircraft. 

Figure  5  shows  an  artist's  conception  of  the  way  in  which  the  ATR  system  would 
closely  integrate  in-flight  training  with  ground-based  training  and  practice  activities. 
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Figure  5 

Artist's  Conception  of  Integration  of  Inflight  and 
Ground-based  Training  via  the  ATR 


Instructor/Student  Associate-Expert  systems  technology  has  potential  for  SUPT 
applications.  It  may  be  possible  to  develop  an  Instructor/Student  Associate  (ISA)  to  aid 
students  as  they  learn  to  fly  in  the  UTD  or  even  in  the  training  aircraft.  Prompting  logic 
within  the  system  would  be  based  upon  instructional  principles  and  a  comparison  between 
the  student's  performance  and  established  performance  criteria.  However,  the  instructor 
would  be  able  to  override  the  ISA  as  needed. 

In  a  fully  automated  mode,  the  ISA  would  provide  aural  cues  or  advice  on  how  to 
correct  the  error  when  a  student  pilot  meets  or  exceeds  allowable  parameters  during  a 
maneuver.  In  a  manual  mode,  the  IP  or  student  could  query  the  ISA  for  advice  on  a 
particular  maneuver  that  has  just  been  completed  or  is  about  to  be  attempted.  The  ISA 
would  be  a  voice-activated  system  to  provide  minimal  distraction  and  would  use  student 
proficiency  profiles  to  adjust  advice  based  on  individual  ability  and  level  of  skill. 

The  ISA  would  require  extensive  development  effort.  Substantial  analysis  and 
modeling  of  training  scenarios  would  be  required  to  identify  and  define  a  suitable  universe 
of  instructional  problems.  Problem  solutions  with  corresponding  instructional 
prescriptions  would  also  require  identification,  test,  and  validation.  Training  and 
psychology  literature  provides  some  general  foundations  for  the  instructional  associate,  but 
the  extent  to  which  an  automated  expert  can  be  designed  to  recognize  and  deal  with 
ongoing  complex  flight  tasks  remains  to  be  determined.  While  such  a  system  could 
potentially  address  many  pilot  training  challenges,  the  technical  risks  inherent  in  such  an 
endeavor  would  be  substantial. 

A  proof-of-concept  videotape  was  developed  as  a  supplement  to  this  report 
(Mattoon  &  Gagel,  1995)  and  demonstrates  how  each  technology  may  enhance  training 
capabilities  and  increase  flexibility  in  SUPT  environments. 


Instructor  Pilot  Feedback  on  Study  Analysis  and  Results 

Following  the  investigation  team’s  analysis  of  findings,  there  was  interest  in 
determining  the  level  of  agreement  among  instructor  pilots  regarding  the  team’s  findings 
about  challenges/problems  found  in  SUPT  as  well  as  technical  solutions  as  proposed. 
After  the  team  had  folly  briefed  the  instructors  on  the  findings,  a  questionnaire  was 
presented  to  a  number  of  the  IPs  who  participated  in  the  original  interviews. 

For  each  of  the  six  training  problem  areas  identified,  the  IPs  were  asked  to  indicate 
the  "degree  of  seriousness"  of  each  problem.  They  were  also  asked  to  indicate  the  degree 
to  which  they  thought  the  proposed  technology  solutions  would  help  overcome  each  of  the 
problems  identified. 

Specific  instructions  contained  in  the  questionnaire  for  rating  the  seriousness  of 
problem  areas  were  as  follows: 
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“Please  rate  each  of  the  six  problem  areas  below  in  terms  of  its  degree  of  seriousness 
in  affecting  SUPT  trainee  performance  and  ability  to  successfully  complete  training. 

(0  =  unsure,  1  =  not  serious,  2  =  somewhat  serious,  3  =  very  serious)” 


Mean  ratings  by  problem  area: 

Position  interpretation 

2.8 

T-37  overhead  traffic  landing  pattern 

2.6 

Formation  and  fluid  maneuvering 

2.6 

Fix-to-fix  navigation 

2.3 

IP  continuation  training 

2.2 

Low  altitude  flying 

2.0 

Instructions  on  rating  the  probable  effectiveness  of  the  modeling/simulation  solutions  were 
recorded.  (Terms  used  in  the  questionnaire  varied  slightly  from  those  mentioned  above, 
but  the  content  of  technology  descriptions  was  essentially  the  same): 

“Please  rate  each  of  the  modeling/simulation  solutions  in  terms  how  effective/useful 
you  think  it  would  be  for  solving  SUPT  problems  experienced  by  students  and  IPs.  (0 
=  unsure  of  usefulness,  1  =  not  useful,  2  =  useful,  3  =  very  useful)" 


Mean  ratings  by  technology  solution: 


Aeronautical  Training  Recorder 

3.0 

Video  media 

2.8 

Instructional  simulation 

2.8 

DART 

2.8 

Large  classroom  monitors) 

2.6 

3-D  Displays 

2.6 

Unit  Training  Device 

2.6 

Improvements  in  CAI 

2.4 

Simulator  Networking 

2.1 

Electronic  Rneepad 

2.0 

(previously  referred  to  as  PET) 


These  results  generally  indicate  that  instructor  pilots  concurred  with  the  challenges 
and  technology-based  solutions  identified  by  the  laboratory  investigators.  IPs  rated  all  of 
the  identified  training  problem  areas  as  “serious”  to  “very  serious”  in  SUPT.  All  of  the 
proposed  technology  solutions  were  rated  as  “useful”  to  “very  useful”  for  application  to 
these  training  challenges. 
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SUPT  INVESTMENT  STRATEGY  ROADMAPS 


The  roadmaps  in  this  section  show  the  time  frames  and  estimated  costs  associated 
with  developing  and  procuring  the  necessary  quantities  of  M&S  technologies  described  in 
this  report  for  implementation  in  SUPT.  The  program  time  frameextends  through  the  year 
2020.  The  assumption  is  that  procurement  of  any  of  these  M&S  systems  could  not  begin 
before  1998  due  to  funding  programming  regulations.  Armstrong  Laboratory  has  R&D 
planned  to  support  many  of  these  efforts.  Laboratory  work  is  shown  where  it  is  planned. 
As  with  any  planning  document,  various  considerations  may  warrant  AETC  procuring 
something  less  than  the  full  slate  of  M&S  technologies  recommended.  AETC  already  has 
some  funding  programmed  in  anticipation  of  some  of  the  technologies  shown  in  the 
roadmaps.  Therefore,  not  all  of  the  funds  required  to  procure  roadmap  technologies  would 
need  to  be  new  program  money.  It  is  also  possible  that  technologies  unknown  at  present 
may  emerge  in  the  next  10  or  15  years  that  could  obviate  selection  of  some  technologies 
recommended  in  this  report. 

The  investment  roadmaps  are  based  on  evaluations  of  current  M&S  technologies  of 
highest  merit  for  intended  purposes.  While  cost  estimates  have  been  included,  they  should 
be  considered  in  view  of  the  character  and  rapid  growth  of  these  technologies.  Estimates 
combine  current  knowledge  of  trends  and  long-term  forecasts  for  technologies. 
Breakthroughs,  although  not  predictable,  must  be  taken  into  account  as  possibilities  due  to 
the  nature  of  this  sector  of  industry.  For  these  and  other  reasons,  we  have  been  interested 
in  developing  a  cost-benefit  model  that  eventually  would  reside  at  AETC  Headquarters. 
Such  a  tool  would  help  AETC  model  hypothetical  applications  of  specific  technologies  to 
better  estimate  training  benefits  potential.  A  preliminary  model  toward  this  objective  is 
contained  in  Appendix  B. 

An  underlying  assumption  in  recommending  these  technologies  is  that  technology 
and  sound  methodology  are  co-dependent.  Therefore,  technology  investment  must  involve 
substantial  methodological  effort  including  training  analysis,  design,  and  technology 
transition.  These  elements  will  be  needed  to  ensure  effective  integration  of  modeling  and 
simulation  within  existing  programs. 

Roadmaps  are  organized  by  technology  development  areas,  so  each  major  roadmap 
may  involve  R&D  for  several  of  the  technologies  described  above  which  may  be  applied  to 
multiple  areas  of  SUPT  training.  Each  roadmap  is  organized  as  follows: 

1 .  Title  of  technology  effort  and  fiscal  year  ribbon  shown  at  the  top; 

2.  Title  or  type  of  subtechnologies  shown  in  boldface  atop  grouped  time  ribbons; 

3.  AL/HRA  lead-in  technology  development  shown  in  shaded  ribbons; 

4.  Specific  milestones  shown  on  charts  as  numbered  triangles  (annotation  of 
triangles  shown  in  left  margin;  and 

5.  Cost  (in  $M)  indicate  estimated  dollars  for  development,  and  operation  and 
maintenance  (O&M)  phases  for  each  technology. 
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Figure  6,  SUPT  Training  Technology  Investment  Strategy,  presents  an  integrated 
picture  of  recommended  technology  investments  for  SUPT.  The  development  and 
application  of  these  technologies  is  intended  to  provide  highly  capable  ground  training 
technology  for  all  SUPT  bases.  Introductory  information  on  each  major  technology 
appears  immediately  below,  and  detailed  descriptions  of  each  major  area  are  contained  in 
each  of  the  subsequent  roadmaps  (Figures  7-10)  and  their  accompanying  explanations. 

Advanced  Simulation  Technologies.  Simulator  training  will  have  high  transfer 
value  because  of  the  superior  fidelity  and  currency  with  training  aircraft  and  flight 
environments.  These  simulation  devices  will  enable  students  to  fully  rehearse  training 
sorties  prior  to  entering  the  aircraft.  It  is  reasonable  to  expect  that  pilots  will  achieve 
current  proficiency  standards  sooner  than  at  present  through  efficiencies  engendered  by 
these  technologies.  It  is  also  possible  that  higher  student  proficiency  levels  can  evolve. 
Simulators  will  be  affordable  in  all  training  squadrons  and  therefore  available  to  students 
when  needed,  precluding  scheduling  delays  and  generally  enhancing  program  efficiency. 

Database  Modeling.  Conceptually,  the  training  database  is  intended  as  the 
functional  center  of  all  modeling  and  simulation  resources  in  SUPT.  Instructors,  students, 
and  training  managers  will  use  this  central,  or  hub,  database  to  access  and  manage  ground 
training  resources  and  student  training  records. 

SUPT  Simulation  Training  Laboratory.  The  training  laboratory,  as 
conceptualized  here,  represents  a  new  training  component  within  SUPT.  It  is  intended  to 
aid  students  make  the  transition  from  knowledge  of  procedures  to  usable  flying  skills.  The 
laboratory  is  intended  to  provide  an  environment  for  the  development  of  basic  hands-on 
stick  and  throttle  (HOT AS)  skills. 

Electronic  Classroom.  The  objective  of  this  program  will  be  to  modernize  the 
academic  learning  environment  through  the  integration  of  state-of-the-art  instructional 
technology.  The  infusion  of  new  technologies  at  this  level  of  training  is  intended  to 
provide  instructors  with  more  efficient  tools,  to  accelerate  student  knowledge  acquisition, 
and  to  improve  the  transfer  of  academics  to  subsequent  training  phases. 

Figure  7,  Advanced  Simulation  Technologies.  This  roadmap  describes  the 
integration  of  technologies  to  provide  advanced  flight  simulators  for  SUPT.  Ongoing 
laboratory  R&D,  identified  in  the  shaded  boxes,  will  provide  the  technology  base  for  each 
of  these  programs.  The  Unit  Training  Device  (UTD)  will  provide  high  fidelity  cockpits  (2 
each,  for  20  SUPT  squadrons).  Terrain  models  and  databases  for  modeling  environments 
of  up  to  six  training  bases  are  included.  DART/HMD  R&D  will  provide  the  technology 
base  for  the  visual  system  to  be  combined  with  the  UTD.  This  program  is  shown  in  two 
technology  phases,  one  for  the  DART  and  one  for  the  follow-on  helmet-mounted  display 
technology.  Thus,  the  DART  wraparound  cockpit  display  screen  eventually  will  be 
replaced  with  helmet  displays  as  a  means  to  keep  abreast  of  the  latest  display  technology. 
The  fourth  component  is  an  advanced  Instruction  Support  System  (ISS)  for  use  by 
simulator  instructors  or  by  individual  students  in  the  UTD  cockpit.  R&D  is  currently 
ongoing  at  the  laboratory  to  underpin  this  program. 
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SUPT  Training  Technology  Investment  Strategy 
Overview  Roadmap 
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Figure  6 

SUPT  Training  Technology  Investment  Strategy 
Overview  Roadmap 


Advanced  Simulation  Technologies 
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Figure  7 

Advanced  Simulation  Technologies 


The  basic  component  of  simulation  training  is  the  UTD,  a  high  fidelity  cockpit 
simulator.  Trainer  cockpits  will  be  functionally  equivalent  and  will  remain  current  with 
the  training  aircraft  during  their  life  cycle.  Aircraft  aerodynamics,  engines,  subsystems, 
avionics,  and  aircxraft  control  characteristics  will  all  be  accurately  modeled  in  UTD 
software.  _  This  portion  of  modernization  envisions  eventual  integration  of  UTD/DART 
systems  as  networks  of  simulators  within  SUPT  squadrons. 

Because  of  the  substantial  lead-in  development  already  accomplished  by  AL/HRA 
(through  FY2001),  the  time  and  cost  of  UDT  prototyping  for  SUPT  purposes  will  be 
minimal  with  system  completion  projected  for  FY2003.  UTD  cockpit  prototyping  and 
production,  and  UTD  terrain  modeling  are  shown  as  separate  timelines  with  their 
associated  milestones.  UTD  software  will  contain  terrain  models  and  training  databases 
for  local  areas  and  ranges  for  each  squadron  location.  Databases  developed  for  UTDs  will 
be  subsequently  integrated  with  the  SUPT  hub  database  in  each  squadron  as  required. 
Prototyping  and  production  phases  are  projected  for  completion  by  FY2002.  The  DART 
effort  will  include  installation  on  UTDs  and  is  phased  to  coincide  with  the  UTD  program. 
The  DART  will  complement  UTD  cockpits  with  high  fidelity  visual  environments.  DART 
development  will  include  an  advanced  image  generator  to  furnish  a  visual  environment 
with  sufficient  quality  to  support  full  mission  rehearsal  for  all  syllabus  events.  As  this 
technology  is  upgraded,  enhancements  will  transition  to  SUPT  during  the  development 
period  shown,  through  FY2002.  The  roadmap  shows  an  approximate  three-year  hiatus  in 
visual  technology  for  UTDs  occurring  between  DART  and  HMD  technology.  The  gap 
reflects  a  maturation  period  for  the  omnidisplay  technology.  The  omnidisplay  will  provide 
very  high  resolution,  wide  field  of  view,  and  other  display  characteristics  superior  to  the 
DART,  but  in  an  HMD  format.  O&M  periods  for  UTD,  DART,  and  HMD  are  projected  to 
coincide  with  integration  of  operating  ysstems  and  will  extend  through  FY2020. 

This  technology  will  be  incorporated  in  UTDs  as  the  successor  to  what  is  now 
called  the  instructor/operator  station,  or  IOS.  It  will  be  based  upon  current  R&D  which 
will  extend  through  FY99.  The  ISS  will  be  capable  of  more  sophisticated  and  direct 
intervention  in  the  training  process,  making  the  student  less  dependent  upon  instructors 
during  simulator  training  activities.  Instructional  capabilities  will  include  advanced 
computer  graphics,  auditory  cueing  and  feedback,  and  direct  tutorial  dialog  with  the 
student.  The  ISS  will  interface  directly  with  the  SUPT  hub  database.  The  prototyping  and 
production  phase  is  projected  for  completion  by  FY2003  with  milestone  events  as  depicted. 

Figure  8,  Database  Modeling  and  Development,  presents  the  timeline  and  cost 
estimates  for  this  technology.  In  the  SUPT  application,  the  central  or  hub  database  will  be 
accessible  via  personal  computers  (or  portable  electronic  trainers  [PETs]),  electronic 
classrooms,  training  laboratories,  and  simulators,  for  use  with  these  systems.  The 
Aeronautical  Training  Recorder  (ATR),  described  in  this  roadmap,  will  also  interface  with 
the  database.  The  database  will  contain  a  host  of  data  models  for  use  in  training.  Libraries 
of  practice  scenarios,  ranging  in  complexity  from  basic  hands-on  cockpit  management 
tasks  to  entire  aircraft  training  missions  will  be  accessible  to  students  via  appropriate 
training  devices  for  given  levels  of  skill.  The  database  will  also  contain  student  records 
and  current  status  in  the  program.  Proficiency  profiles,  based  on  student  training 
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Figure  8 

Database  Modeling  and  Development 


performance  data,  will  be  used  to  diagnose  individual  training  needs  and  activities. 
Eventually,  the  database  will  provide  automated  training  guidance,  integrating  student 
training  records  and  providing  training  guidance.  The  central  database  will  improve  time 
and  training  for  gorund  training  resource  management.  Implementation  of  the  student 
performance  data  system  will  enable  students  to  build  knowledge  and  flying  skills  more 
rapidly  and  thoroughly.  It  will  provide  managers  with  much  more  explicit  information  for 
cost  benefits  determinations  and  program  accountability.  An  overview  of  benefit-cost 
analysis  applied  to  the  R&D  in  this  report  is  provided  in  Appendix  B. 

IP  Data  Entry  System.  This  addition  to  student-training  functions  will  provide  the 
flight  instructor  with  a  convenient,  effective  means  to  annotate  student  performance  during 
aircraft  sorties.  The  model  developed  for  SUPT  will  be  preceded  by  preliminary  R&D  at 
AL/HRA,  to  be  completed  by  FY2000.  Prototyping  and  production  will  begin  in  FY2001 
and  be  completed  by  FY2005,  with  O&M  projected  through  FY2020.  This  system  will  be 
the  first  enhancement  to  the  hub  database,  however,  its  development  will  be  accomplished 
almost  concurrently  with  the  central  system  itself. 

Aeronautical  Training  Recorder  (ATRL  Database  development  will  also  include 
development  and  integration  of  the  ATR  and  Flying  Training  Expert  System  as  phased 
system  enhancements.  The  program  is  based  on  installation  of  systems  aat  four  user 
locations  (SUPT  bases).  ATR  data  recording  will  include  IP  Data  Entry  System  inputs  as 
designated  by  the  instructor  for  debriefs.  The  net  effect  sought  from  ATR  implementation 
is  reduced  time  to  student  proficiency  in  the  aircraft.  As  with  other  technologies,  an 
advanced  period  of  R&D  by  AL/HRA  (to  be  completed  by  FY2001)  will  precede  device 
prototyping  and  production  (to  be  completed  by  FY2009).  The  development  and 
integration  of  this  technology  will  extend  over  a  considerable  time  period  and  envisions 
gradual  system  upgrades  as  state-of-the-art  advances  in  expert  systems  become  operational. 

O&M  for  the  program  will  begin  when  the  basic  system  goes  on  line  in  SUPT,  about 
FY2005  and  will  extend  through  FY2020. 

Instructor/Student  Associate  flSAL  Finally,  this  roadmap  shows  a  program  for  ISA 
development.  This  system  is  intended  for  use  in  the  simulator  and  possibly  the  training 
aircraft.  A  five-year  laboratory  lead-in  period  is  envisioned  beginning  in  FY2002  as  a 
specific  application  of  maturing  expert  systems  technology  anticipated  within  industry. 
Lead-in  R&D  will  focus  on  SUPT  training  tasks  in  order  to  provide  needed  databases. 

Figure  9,  SUPT  Training  Laboratory.  Lead-in  R&D  for  the  training  laboratory 
includes  part-task  training  and  instructional  simulation  concepts  developed  within  the 
laboratory  and  elsewhere,  combined  with  current  microcomputer  technologies.  These 
efforts,  ongoing  currently,  will  be  the  foundation  for  the  design  and  integration  of  the 
training  laboratory  as  an  innovation  into  the  SUPT  program.  It  is  possible  that  training 
capabilities  for  all  basic  cockpit  procedures  and  navigational  skills  can  be  built  into  a 
single,  multifunction  HOTAS  system  which  could  shorten  development  of  the  prototype  to 
two  years.  The  production  phase  is  projected  for  a  three-year  effort  to  completely  equip  all 
SUPT  squadrons.  The  training  laboratory  will  contain  medium-level  fidelity  cockpits  to 
train  HOTAS  and  basic  navigation  instrument  skills.  The  complementing  visual  display 
systems  will  be  developed  concurrently  with  the  cockpits.  Display  technology  may  consist 
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Figure  9 

SUPT  Training  Laboratory 


of  either  DART  or  HMD  technology  and  digital  image  generators.  Costs  shown  for  these 
systems  include  integration  with  cockpits.  Commercial,  off-the-shelf  components  for 
visual  displays  and  image  generators  will  be  assembled  and  integrated. 

The  training  laboratory  will  require  two  additional  developments.  The  first  is 
training  software  which  includes  modification  and  integration  of  available  UTD  software 
with  HOTAS  cockpits.  The  dual  implementation  will  save  development  costs  and  increase 
training  efficiency.  O&M  software  costs  include  currency  upgrades  throughout  the  life 
cycle.  Instructional  software  development  will  also  be  included  in  this  effort.  Flight 
scenarios  developed  for  the  simulation  training  laboratory  will  incorporate  instructional 
strategies  and  feedback  techniques  to  aid  student  skill  acquisition.  All  software  for  the 
training  laboratory  will  be  managed  by  the  SUPT  hub  computer  system.  The  second 
development  is  an  enhancement  for  the  training  laboratory.  The  goal  will  be  to  produce  a 
surrogate  instructor  pilot  (expert  system)  specifically  for  use  in  the  training  laboratory. 
This  system  is  intended  to  obviate  the  need  for  live  instructors  in  the  training  lab. 

Figure  10,  Electronic  Classroom.  This  section  includes  technology  upgrades  for 
CAI  and  classroom  training  hardware  (LSDs,  personal  computers,  and  desktop  training 
stations).  Modernized  classrooms  will  integrate  state-of-the-art  instructional  technology 
with  syllabus  content.  Students  will  engage  in  individual  training,  team  training,  and 
instructor/student  coaching  activities  using  these  devices.  Students  will  access  the  hub 
computer  system  via  personal  computers  to  monitor  their  proficiency  profiles  and  advance 
according  to  individual  progress  and  on-line  skill  assessments. 

AL/HRA  plans  the  development  of  prototype  electronic  classroom  technologies 
during  FY96  to  FY98.  This  R&D  effort,  entitled  the  Aircrew  Integrated  Learning  System 
(AILS),  will  provide  needed  analytical  and  developmental  foundations  to  support  SUPT 
academics.  Hardware  for  this  system  is  envisioned  as  commercial,  off-the-shelf  media. 
Courseware  development  will  comprise  the  major  portion  of  this  effort.  System 
prototyping  and  integration  is  projected  for  completion  in  FY2004  with  O&M  beginning 
then  and  extending  thorugh  FY2020. 

Costs  shown  in  the  two  lower  boxes  in  this  roadmap  represent  development  of  a 
library  of  courseware  and  academics  media.  We  assume  AETC's  media  production 
resources  will  be  available  to  satisfy  requirements  for  these  materials.  Therefore,  no  costs 
are  included  in  this  roadmap  for  development  of  academic  media  materials. 

Developing  electronic  classroom  technology  for  SUPT  assumes  modernizing  40 
classrooms  (10  classrooms  x  4  SUPT  bases).  Basic  components  (commercial,  off-the-shelf 
items)  to  equip  the  environment  are  shown  in  the  roadmap.  Systems  are  shown  separately, 
but  in  concurrent  development  beginning  in  FY99  and  extending  through  FY2004.  Costs 
shown  for  component  systems  include  time  and  costs  required  for  selection,  acquisition, 
integration,  and  testing  as  "turnkey"  systems  for  SUPT  classrooms.  The  6  ft  x  8  ft,  high- 
resolution  LSDs  will  be  integrated  with  image  generators  for  computer  animation  and 
simulation.  All  students  will  be  issued  PETs  for  continuous  use  throughout  the  training 
cycle  and  access  to  desktop  training  stations  equipped  with  HMDs  at  strategic  locations 
throughout  the  training  base.  PET  training  stations  will  network  with  other  PETs,  LSDs, 
UTDs  and  the  SUPT  hub  computer  to  enable  students  to  engage  in  self-monitoring  of 
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Figure  10 

Electronic  Classroom 


performance;  to  individualize  training  activities  according  to  the  student's  current  level  of 
ability  and  training  experience;  and  to  provide  for  team  training  activities  that  involve  two 
or  more  students  or  student/instructor  teams.  We  anticipate  that  frequent  upgrades  will  be 
required  to  maintain  electronic  technology  currency.  Therefore,  it  seems  reasonable  that 
students  would  retain  PETs  following  SUPT  completion,  with  PET  replacement  absorbed 
as  recurring  upgrade  costs  ($2K/student).  This  would  preclude  accumulation  of  obsolete 
equipment,  maintain  state-of-the-art  systems,  and  benefit  students  as  they  enter  CCTS. 

Classroom  instructors  will  control  presentations  and  otherwise  manage  training 
activities  via  a  classroom  control  console.  Instructors  will  monitor  student  learning  and 
modify  content  as  needed  to  insure  achievement  of  course  objectives.  Instructors’ 
activities  will  include  teaching  and  coaching  certain  individual  training  events,  monitoring 
and  advising  students  on  their  use  of  the  proficiency  profile  system  and  automated  trainers, 
and  managing  team  training  activities. 


CONCLUSIONS 


We  believe  the  modeling  and  simulation  technologies  described  in  this  report  can 
improve  SUPT  in  general  and  the  six  identified  training  challenge/problem  areas  in 
particular.  While  these  six  areas  were  the  ones  most  often  mentioned  during  the 
interviews,  they  were  not  the  only  areas  of  need  described  by  the  IPs.  It  is  important  to 
recognize  that  acquiring  these  technologies  would  benefit  many,  if  not  all,  other  areas  of 
the  SUPT  syllabi  as  well. 

A  key  issue  is  how  these  technologies  would  be  integrated  into  the  existing  SUPT 
syllabi  given  the  fullness  of  existing  programs.  We  assume  syllabi  length  would  not  be 
expanded  merely  to  accommodate  new  training  media.  We  are  confident  that  all  of  the 
technologies  recommended  can  generate  sufficient  training  efficiency  to  more  than  offset 
the  R&D  efforts  needed  for  their  implementation  within  AETC  programs.  This  would 
result  from  replacing  less  efficient  existing  training  materials  and  methods  and  reducing 
training  task  loads  via  new  M&S  tools. 

Another  important  issue  is  the  potential  created  by  these  innovations  to  improve 
student  progress  through  the  training  cycle  and  their  level  of  pilot  ability  on  graduation 
from  the  program.  Our  analysis  leads  us  to  believe  that  these  innovations  could  enable 
SUPT  to  move  beyond  the  traditional  "lock-step"  pattern  and  produce  even  higher  levels  of 
pilot  skill.  Rather  than  basing  individual  student  advancement  exclusively  upon  an  average 
or  group-determined  pace,  students  could  advance  individually  as  they  meet  prescribed 
performance  criteria.  By  this,  we  are  not  advocating  total  self-paced  progress,  because 
experience  with  that  method  shows  some  trainees  may  finish  so  fast  or  slow  that  program 
management  processes  cannot  be  accommodated.  Rather,  students  would  progress 
individually  within  blocks  of  group-oriented  instruction.  When  a  student  finishes  a  block 
before  others,  advanced  pilot  practice  activities  can  be  employed  until  the  rest  of  the  group 
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is  ready  for  the  next  block.  Thus,  students  with  outstanding  aptitude  would  continue 
striving  to  reach  their  full  potential  instead  of  simply  waiting  for  the  majority  to  catch  up. 
Thus,  the  proficiency-advancement  method  would  not  only  ensure  that  individual  students 
receive  optimal  training  at  the  time  they  need  it,  but  also  would  help  identify  and  promote 
the  most  capable  pilots. 

This  report  has  described  Phase  I  of  AL/HRA's  M&S  R&D  effort.  Phase  II  will 
apply  the  same  analytical  methodology  to  other  AETC  flying  training  programs.  Phase  II 
will  target  "schoolhouse"  applications  within  selected  CCTS.  AL/HRA  has  considerable 
experience  with  such  units  and  believes  the  methodology  used  for  Phase  I  can  be  applied 
effectively  for  Phase  II.  In  addition,  we  believe  that  many  of  the  same  M&S  technologies 
identified  for  SUPT  may  also  benefit  CCTS. 
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APPENDIX  A 


AETC  QUESTIONNAIRE  MATERIALS 
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AIR  FORCE  MODERNIZATION  PLAN 
AETC  Flying  Training 
Mission  Area  Plan  (MAP) 


Date: 


Unit: 


CURRENT  PILOT  DATA: 

Rank: _  Rating: _  Total  Flight  Hours: _ 

HISTORY  (Prior  Aircraft) 

Aircraft  No.  Hours/Ratings  Command/Organization 


1 .  What  qualification  level  were  you  prior  to  this  assignment? 

a.  Mission/Package  Commander 

b.  Aircraft  Commander 

c.  First  Pilot 

d.  Flight  Lead 

e.  Evaluator  Pilot  (Stan/Eval) 

f.  IP 

g.  Other _ 

2.  Where  was  UPT  instructor  on  your  duty  preference  list? 

a.  First 

b.  Near  the  top 

c.  Near  the  bottom 

d.  UPT  Instructor  was  not  on  my  list  of  duty  priority  choices 

3 .  How  long  have  you  been  a  UPT  IP? 

Arrival  Date _ _ _  Today’s  Date _ 


4.  Do  you  enjoy  being  an  IP? 

YES/NO 

Why? 
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5.  From  what  major  category  of  platform  do  you  believe  the  most  effective  instructor 
pilots  come? 


a.  Fighter 

b.  Bomber 

c.  Tanker/Transport 

d.  Other 

e.  It  doesn’t  matter 
Why? 


6.  Is  it  possible  to  predict  which  students  will  have  difficulties? 

YES/NO 
If  YES,  How? 


7.  Is  it  possible  to  predict  where  the  problems  will  occur? 

YES/NO 
If  YES,  How? 


8.  What  can  students  do  that  might  remove  or  reduce  problems  that  are  typically 
encountered? 


9.  Where  do  you  believe  the  major  training  systems/resources  “choke  point”  exists  in  the 
training  pipeline? 


Why? 


10.  Where  do  you  believe  the  major  student/leaming  training  “choke  point”  exists  that  is  a 
student  problem  area? 
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1 1 .  What  method,  medium,  or  technique  do  you  use  to  get/keep  the  student  on  track  and 
moving  toward  graduation? 


12.  If  you  believe  it  is  a  thinking  problem  for  the  student  (thinking  in  an  aircraft  that 
operates  above  360  kts  for  instance),  what  specifically  do  you  think  is  difficult  for  him/her 
to  understand? 


13.  If  you  believe  that  it  is  a  conceptual  problem,  is  it  that  the  student  has  trouble 
understanding  three-dimensional  spatial  concepts  involved  in  any  block  (instruments, 
contact,  formation,  other,  etc.)? 

YES/NO 
If  YES,  What? 


Why? 


14.  If  you  believe  that  it  is  a  procedural  task  (intercepting  the  ELS,  landing  pattern 
procedures,  etc.)  that  a  student  has  trouble,  what  parts  or  steps  are  most  difficult? 


Why? 


15.  Are  there  deficiencies  in  the  training,  resources,  or  equipment  which  contribute  to  this 
particular  problem? 

YES/NO 
If  YES,  What? 


Why? 
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16.  Instructors  have  observed  that  some  students  just  seem  to  have  to  make  certain  errors 
first— that  is,  to  stumble  through  a  task  once  or  twice  before  they  learn  enough  “about”  the 
task  to  begin  to  learn  the  task  and  perform  correctly. 

a.  Do  you  agree?  YES/NO 

b.  If  so,  are  there  any  examples  from  the  syllabus  you  feel  fit  this  description? 


c.  Why  do  you  think  this  happens? 


17.  Students  typically  screw  up  learning  some  tasks.  What  three  are  done  the  most? 

a. 

b. 

c. 


18.  What  method,  medium,  or  technique  do  you  use  in  those  areas  identified  to  get/keep 
the  student  on  track  and  moving  toward  graduation? 


19.  What  areas  of  air  training  need  more  emphasis  (e.g.,  instrument,  contact,  formation)? 


20.  What  areas  of  academic  training  need  more  emphasis  (e.g.,  aero,  meteorology, 
instruments,  etc.)? 


21 .  Are  there  any  areas  of  this  simulator  portion  of  the  program  that  need  more  emphasis 
(e.g.,  EPs  or  instruments),  or  could  use  enhancements/upgrades? 

YES/NO 
If  YES,  What? 
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22.  From  your  own  experience,  have  you  ever  thought  that  some  students  seem  to  learn 
better  from  certain  types  of  training  media,  methods,  or  devices? 


YES/NO 

If  YES,  What  type? 


Why? 


23.  How  many  hours  do  you  feel  a  student  should  study  each  day  outside  the  scheduled 
training  hours? 


24.  Should  voluntary  simulator/training  aid  use  after  duty  hours  count  against  crew  rest? 
YES/NO 

25.  Could  you  identify  any  problems  with  IP  (PIT)  training? 

YES/NO 
If  YES,  What? 

26.  What  would  you  do  to  change  Air  Force  UPT? 

Why? 
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FOR  SCHEDULE  WRITERS/PLANNERS 


1 .  What  kind  of  training  (OJT,  etc.)  program  do  schedule  writers  have? 


2.  Do  some  planners  seem  to  write  a  more  efficient  schedule  than  others? 

YES/NO 
If  YES,  Why? 


3.  Considering  the  normal  obstacles  encountered  in  the  training  process,  what  is  the 
greatest  difficulty? 


a.  Weather 

b.  Aircraft  availability 

c.  Instructor  availability 

d.  Student  availability/readiness 

e.  Other 


4.  Do  you  use  any  type  of  electronic  systems/computerized  scheduling  programs  to  assist 
you  in  your  scheduling  process? 

YES/NO 
If  Yes,  What? 


5.  Do  you  believe  that  developing  a  NEW,  or  user  friendly  computerized  scheduling 
system  would  be  of  any  benefit  to  your  tasking? 

YES/NO 
If  No,  Why  not? 


6.  What  scheduling  functions  give  you  problems? 
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7.  What  training  information  (e.g.,  student  does  DNIF,  academic  completions  for  the  day, 
sortie  completions)  do  you  need  to  accomplish  your  job? 


8.  Are  your  needs  for  information  being  met  well? 
YES/NO 


9.  What  impact  does  alternating  early  week/late  week  have  on  your  scheduling  of 
students? 

a.  None 

b.  Very  little 

c.  Some 

d.  Much 

e.  Very  much 

10.  How  drastically  does  “standing  down”  a  student  impact  training  continuity  and  the 
scheduling  workload? 

a.  None 

b.  Very  little 

c.  Some 

d.  Much 

e.  Very  much 


11.  If  you  use  electronic  scheduling  devices,  how  does  the  computer  handle  “standing 
down”  a  student? 
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APPENDIX  B 

BENEFIT-COST  ANALYSIS 
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THE  DEVELOPMENT  OF  BENEFIT-COST  ESTIMATION  METHODS  FOR 
TECHNOLOGICAL  IMPROVEMENTS  IN  UNDERGRADUATE  PILOT  TRAINING 

William  C.  Moor  and  Barbara  G.  Nelson 


Introduction 


The  Aircrew  Training  Research  Division  of  the  Armstrong  Laboratory  (AL/HRA)  is  actively 
engaged  in  research  on  the  development  of  an  enhanced  and  improved  undergraduate  pilot 
training  program.  The  U.S.  Air  Force  (USAF)  desires,  insofar  as  possible,  that  proposed  capital 
expenditures  be  based  on  a  benefit-cost  comparison  among  all  competing  alternatives  (Dept  of  the 
Air  Force,  1988).  The  purpose  of  this  project  is  to  develop  a  method  of  applying  the  technique  of 
benefit-cost  analysis  to  the  evaluation  of  proposed  technology-based  systems  which  are  intended 
to  provide  improved  undergraduate  pilot  training  (UPT)  for  the  USAF.  A  difficulty  exists 
because  even  though  benefit-cost  analysis  is  a  widely  documented  and  well  understood  technique, 
the  mechanisms  for  establishing  and  measuring  benefits  are  not  universally  defined  and  must  be 
developed  for  each  specific  application. 

There  is  a  general  consensus  in  the  economic  literature  that  a  benefit-cost  analysis  is  appropriate 
for  any  government  program  that  involves  an  impact  on  a  segment  of  the  population  (Maciariello, 
1975;  Pearce,  1983;  Steiner,  1980).  Enhancing  the  UPT  for  the  USAF  meets  this  criterion. 
Therefore,  alternative  methods  of  providing  such  training  should  be  amenable  to  this  analysis. 

There  is  also  a  general  agreement  (Maciariello,  1975;  Pearce,  1983;  Oxenfeldt,  1979)  that  a 
benefit-cost  analysis  includes: 

1.  A  specification  of  the  goal(s)  or  objective(s)  which  the  programs  are  intended  to  achieve. 

2.  An  enumeration  and  definition  of  the  alternatives  which  are  proposed  as  mechanisms  for 
reaching  these  goal(s). 

3.  A  definition  of  the  means  of  evaluating  the  benefits  derivable  from  each  alternative.  This 
includes  the  definition  of  the  means  of  converting  benefits  into  quantitative  (dollar  measurable) 
terms. 

4.  A  definition  of  the  means  of  evaluating  the  costs  required  to  implement  each  alternative. 

5.  A  completed  analysis  showing  the  computed  incremental  benefit/cost  (B/C)  ratios  for  all 
alternatives  being  evaluated.  Alternatively,  incremental  net  benefits  may  be  used  as  a  basis  for 
evaluating  alternatives). 
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Application:  Goal 


The  specific  goal  for  the  alternatives  to  be  evaluated  in  this  review  is  "to  enhance  the  training  of 
pilots  through  an  improved  undergraduate  training  program."  Therefore,  the  benefit-cost  analysis 
should  provide  a  means  of  comparing  any  alternative  which  has  the-  capability  of  achieving  this 
goal. 

Application:  Alternatives 

The  Aircrew  Training  Research  Division  of  the  Armstrong  Laboratory  has  developed  a  Problem 
Area/Solution  Matrix  in  which  the  areas  of  highest  utility  have  been  estimated.  Table  1  presents 
this  matrix  and  indicates  specific  areas  for  which  preliminary  benefits  estimates  have  been 
developed.  With  the  rapid  technological  improvements  made  in  microcomputers  and  vision 
systems,  it  is  possible  to  construct  relatively  inexpensive,  high  performance  simulation  systems 
and  databases  that  could  be  used  to  meet  many  of  the  training  enhancement  needs  of  the  Air 
Force.  The  difficulty  would  be  to  select  the  system,  or  systems,  which  would  provide  the  greatest 
training  benefit  at  a  cost  within  budgetary  limitations.  Providing  a  benefit  model  that  would  be 
applicable  to  the  possible  solution  systems  and  technologies,  either  currently  available  or 
proposed,  would  help  to  alleviate  this  difficulty.  The  focus  of  this  evaluation  would  be  on  an  Air 
Force-wide  implementation  of  simulation  systems  rather  than  the  design  or  use  of  any  single 
system. 

Application:  Evaluating  benefits 

A  Benefits  Estimation  model  has  been  developed  which  demonstrates  a  complete  method  of 
benefit-cost  analysis  of  the  enhanced  training  alternatives  and  provides  a  means  of  computing  the 
values  for  this  analysis  in  a  manner  that  is  very  straightforward  (utilizing  LOTUS  1-2-3 
spreadsheets).  The  model  includes  the  capacity  to  evaluate  and  compare  the  training 
environments  as  an  explicit  element.  The  following  assumptions  have  been  made  in  the 
development  of  the  benefits  estimation  model: 

1 .  Simulation-based  systems  would  be  implemented  as  pairs  in  the  UPT  squadrons. 

2.  Although  implemented  in  pairs,  the  benefits  estimation  is  based  on  a  single  system. 

3.  A  level  of  skill  can  be  defined  and  measured  for  each  area  and  can  be  scaled  to  a  range  of 
0.0  to  1.0. 

4.  A  minimum  number  of  aircraft  hours  must  be  spent  training  each  skill  (to 
confirm/familiarize/verify  that  the  skill  exists). 

5.  No  reduction  in  total  scheduled  flight  hours  per  trainee  will  occur.  Any  "loss"  of  light  hours 
in  training  on  the  simulator  system  will  be  flown  in  training  other  skills. 
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Table  1 .  Problem  Area/Solution  Matrix 
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=  Not  applicable 


6.  Any  scheduled  flight  hours  “lost”  (due  to  inclement  weather,  maintenance,  etc.)  would  be 
spent  using  the  simulator  system. 

7.  A  level  of  the  retention  of  skill  can  be  defined  and  measured  for  each  area  and  can  be  scaled 
to  a  renge  ofiO.O  to  1.0. 

The  Benefits  Estimation  model  uses  two  approaches  in  determining  the  estimated  benefits. 
Estimated  benefits  could  result  from  Simulator  Networking  which  would  include  the  possible 
solutions  or  technologies  of  Instructional  Simulation,  Simulator  Networking,  DART  Visual  and 
the  T-37,  T-38,  T-l  Multi-Task  Trainer.  The  general  model  for  these  areas  of  benefits  estimation 
is  presented  in  Table  2.  Estimated  benefits  could  also  result  from  Non-Simulator  Based  Training 
Systems  which  would  include  the  possible  solutions  or  technologies  of  Large  Classroom  Monitor, 
3-D  Projections,  Video  Media,  CAI  Improvements,  Electronic  Kneepad  and  the  Aeronautical 
Training  Recorder.  The  general  model  for  these  areas  of  benefits  estimation  is  presented  in  Table 
3.  Table  4  presents  the  computation  equations  used  in  each  of  these  approaches.  In  use, 
estimates  of  the  variables  used  in  these  equations  are  established  by  the  user  with  the  benefits 
estimations  being  computed  automatically.  As  indicated  in  Table  1,  very  rough  estimates  of 
benefits  derivable  for  a  sub-set  of  the  possible  areas  of  technology  application  have  been 
developed  based  on  input  from  HRA  experts.  The  tables  showing  these  estimates  are  presented  in 
Appendix  A  to  this  report. 

Application:  Costs  Estimation 

The  cost  model  was  developed,  as  closely  as  possible,  according  to  the  rules  and  procedures 
currently  in  use  by  the  Air  Force.  The  development  of  the  model  is  more  a  case  of  selecting  than 
defining  or  deriving  the  key  cost  elements  which  must  be  used  in  evaluating  organizational 
alternatives.  Table  5  presents  the  general  cost  model  using  these  cost  elements.  The  information 
was  extracted  from  Air  Force  Regulation  173-15,  Economic  Analysis  and  Program  Evaluation  for 
Resource  Management. 

The  data  for  the  variables  in  the  cost  model  came  from  interviews  with  AL/HRA  personnel  who 
were  very  familiar  with  the  commercial  simulation  device  market  (M.  Thomas,  personal 
communication,  June  27,  1990),  Air  Force  cost  estimating  manuals  (Directorate  of  Engineering 
and  Services,  1988;  Grant  and  Thomley,  1987;  Woolsey,  1987),  and  civil  service  and  military  pay 
rates  (Air  Force,  1990).  The  following  assumptions  have  been  made  in  developing  the  cost 
estimate  model: 

1 .  Each  organizational  alternative  will  be  manned  by  military  and  civil  service  personnel. 

2.  All  buildings  and  equipment  will  be  acquired  new. 

3.  All  computer  based  equipment  will  have  a  definable  but  relatively  short  economic  life. 
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Table  2.  Benefits  Estimation 

Use  of  simulation  systems  to  enhance  training  in  generalized  activities 

NUMBERS  IN  THIS  SHEET  DO  NOT  APPLY  TO  ANY  SPECIFIC  SIMULATOR  SYSTEM 


Estimated  TotaLBenefit  of  1  simulation  system 


-  $1,704,067 


Variables,  Constants  and  Intermediate  Computed  Results  Used  for  Benefit  Estimation 
Variables,  Constants  and  Intermediate  Computed  Results 
Used  for  Benefit  Estimation 


VALUE  RANGE 

NAME  OF  THE  ELEMENT  ASSIGNED  NAME 


Cost  of  the  aircraft  ($/aircraft) 

$8,000,000 

|AIRC$ 

Marginal  cost  of  flying  the  aircraft  ($/hr) 

$1,500 

|AIRC$HR 

Probability  of  loss  of  aircraft  (/hr) 

0.0000002 

|AIRCPROB 

Cost  of  instructor  pilot  time*  ($/hr) 

$48 

mam 

"Cost"  of  the  IP($) 

$10,000,000 

ip$  j 

Cost  of  trainee  time  ($/hr) 

$20 

"Cost"  of  the  trainee  ($/trainee) 

$6,000,000 

TRNEE$ 

Time  spent  in  THIS  training  (hrs/trainee) 

40 

FORM_HR 

transfer  Ratio 

0.3000 

TRANRAT 

Operation  days/week/simulator  (dys/wk) 

5 

DYSJ/VK 

Operation  hours/day/simulator  (hrs/dy) 

16 

HRS_DY 

Operation  weeks/year/simulator  (wks/yr) 

50| 

WKS_YR 

Percent  uptime 

0.9000| 

UP_PER 

Percent  time  the  simulator  is  used  for  THIS  training 

i.ooooj 

FORM_PER 

Total  hours  the  simulator  is  used  for  THIS  training  (hrs) 

3600| 

SIMHRS 

Hours  of  aircraft  time  "lost"  per  year  (hrs/yr) 

lOOj 

AIRC_LST 

Syllabus  hours  for  THIS  training  (hrs/trainee) 

40 

SYLBHRS 

Proportion  of  time  the  instructor  flies  in  same  aircraft  as  trainee 

0.8000 

IP_PROP 

Proportion  of  time  per  sortie  spent  in  THIS  TYPE  OF  training 

0.6000 

SRT_PROP 

Proportion  of  time  the  IP  is  able  to  spend  evaluating  the 
trainee  during  flight 

0.6000 

EVAL_PR 
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Table  2.  Cont'd 

Benefits  Estimation  Results  Using  Equation  Estimators 


ESTIMATED  RANGE 

IDENTIFICATION  OF  EQUATION  AND  EQUATION  BENEFITS  NAME 


Benefit  due  to  aircraft  not  being  destroyed 
+SIMHRS*TRANRAT*AIRC*AIRCPROB 

$1,728 

AIRC$BEN 

Benefit  due  to  trainee  not  being  killed 
+SIMHRS*TRANRAT*TRNEE*AIRCPROB 

$1,296 

TRN$BEN 

Benefit  due  to  IP  not  being  killed 
+SIMHRS*TRANRAT*IP*IP PROP*AIRCPROB 

$1,728 

IP$BEN 

Savings  in  aircraft  use  due  to  simulation 
+SIMHRS*TRANRAT*AIRC  HR 

$1,620,000 

AHRS$BEN 

Benefit  due  to  increase  in  THIS  TYPE  OF  training  time 
+SIMHRS*TRANRAT*SRT PROP*(IP  HR*IP PROP+TRN  HR) 

$37,843 

w 

Benefit  due  to  increased  efficiency  of  use  of  IP  time 
+SIMHRS*TRANRAT*IP _ HR*IP_PROP*FORM_PER 

$41,472 

IP$_EFF 
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Table  3.  Training  Systems  Benefits  Estimation  Worksheet 

Use  of  new  training  systems  to  enhance  training  in  generalized  activities 

VALUES  IN  THIS  SHEET  DO  NOT  REFER  TO  ANY  SPECIFIC  TECHNOLOGY 


Estimated  Total  Benefit  of  a  new  training  system 

$441,826 

Variables,  Constants  and  Intermediate  Computed  Results  Used  for  Benefit  Estimation 

'  • 

VALUE 

RANGE 

NAME  OF  THE  ELEMENT 

ASSIGNED 

NAME 

Level  of  skill  achieved  under  the  current  syllabus  and  conditions 

0.3000 

L C 

Number  of  hours  in  the  current  syllabus  for  training  in  this  skill 

25.00 

H_C 

Proportion  of  instructor  hours/student  hours  -  current  syllabus 

0.5000 

P C 

Cost  of  instructor  pilot  time  ($/hr) 

$48 

l_$ 

Cost  of  trainee  time  ($/hr) 

$20 

s_$ 

Level  of  skill  achievable  with  the  proposed  training  system 

0.5000 

L_N 

Number  of  hours  expected  when  using  the  proposed  system 

20.00 

H_N 

Proportion  of  instructor  hours/student  hours  -  proposed  system 

0.3333 

P N 

Number  of  students  per  year  undergoing  training 

200 

N_S 

Number  of  instructor  hours  current  spent  training  this  skill 

2500 

l_HRS 

Retention  of  skill  under  new  technology 

0.15 

RS_NEW 

Retention  of  skill  under  old  technology 

0.08 

RS OLD 

Benefits  Estimation  Results  Using  Equation  Estimators 

ESTIMATED 

RANGE 

IDENTIFICATION  OF  EQUATION  AND  EQUATION 

BENEFITS 

NAME 

Benefit  due  to  training  efficiency  based  on  student  costs 

86666.66667 

$86,667 

|sTUD_EFF 

Benefit  due  to  training  efficiency  based  on  instructor  costs 

312031.2031 

$312,031 

|INS_EFF 

Benefit  due  to  increased  retention  of  skill 

43128.37534 

$43,128 

jRSKILL 
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Table  4.  Computation  Equations  Used  to  Estimated  Benefits  Resulting  from  Simulator  Networking 
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Benefit  due  to  increase  retention  of  skill  resulting  from  the  use  of  new  learning  technology. 

(Retention  of  skill  with  the  new  technology /Retention  of  skill  with  existing  technology)  X  (Number  of  hours  of  training  required  using  existing  technology/ 
Number  of  hours  of  training  required  using  the  new  technology)  X  ((Proportion  of  instructor  hours  to  student  hours  under  current  syllabus)  /  (Proportion 
of  instructor  hours  to  student  hours  using  the  proposed  system))  X  (Hourly  cost  of  IP  time  +  Hourly  cost  of  student  time)  X  (Total  number  of  students  per  year) 


Table  5.  Specific  Technology  or  System  for  Which  Costs  are  Being  Estimated 


THE  VALUES  ENTERED  IN  THIS  TABLE  DO  NOT  REFER  TO  ANY 
SPECIFIC  TECHNOLOGY  OR  SYSTEM 


Information  extracted  from  AF  Regulation  173-15,  March  1988,  "Economic  Analysis  and  Program 
Evaluation  for  Resource  Management,  Cost  Analysis",  Department  of  the  Air  Force, 

Headquarters  USAF,  Washington  D.C. 

Standard  Terms  and  Areas  Identified  as  those  to  classify  or  describe  categories  of  cost. 


1 .  Life-cycle  Costs  (LCC)  -  Total  Annual  Costs  - 
acquisition  to  disposal  of  a  particular  alternative. 
Discounted  values  for  costs. 

2.  Non-recurring  Costs  -  One  time  costs,  unique  to  a 
specific  phase  of  a  project. 

a.  Research  and  development  costs 

b.  Investment  costs  -  costs  associated  with  the 
acquisition  of  equipment,  real  proterty 
start-up  costs. 

1)  Costs  of  acquisition,  rehabilitation 
or  modification  of  land,  buildings, 
machinery,  equipment,  and  any  long  term 
computer  software  costs. 

2a)  Costs  of  acquisition,  rehabilitation, 
or  modifications  of  other  capital 
items  such  as  furnishings  and 
fittings  required  for  the  project. 

2b)  Cost  of  acquisition,  rehabilitation, 
or  modification  of  high  technology 
equipment  such  as  computers  which 
has  a  very  such  economic  life. 

3)  The  cost  of  plant  rearrangement  and 
tooling  associated  with  the  project. 

4)  The  costs  of  freight  and  insurance 
required  by  the  project. 

5)  The  value  of  nonrecurring  services 
received  from  others. 

6)  The  costs  of  leaseholds  required  for 
the  project. 

7)  Working  capital  and  current  assets  on 
hand  or  on  order. 

8)  Imputed  value  of  existing  Air  Force 
assets  to  be  employed  on  the  project. 


TOTAL  VALUE  OF  ALL  NON-RECURRING  COSTS 


Estimated 

Costs  Notes  and  comments 


$1,987,139 


$1,000,000 


$500,000 


$100,000 
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Table  5.  Contd 


Estimated  Costs 


Notes  and  comments 


3.  Recurring  Costs  -  Annual  costs  required  to 
operate  and  maintain  a  program  or  project, 

a.  Personnel  Costs  -  All  direct  and  indirect 
costs  related  to  both  civilian  and  military 
personnel. 

1)  Gross  pay 

2)  Retirement  and  disability,  health, 
and  life  insurance. 

3)  Sick  leave  and  annual  leave 

4)  Holiday  pay  and  other 

5)  Change  of  station  or  Duty  pay 

6)  Retirement  pay 

7)  Travel,  per  diem,  moving  expenses 

8)  Training. 

b.  Supplies  and  Material 

1)  Directly  consumed 

2)  Transportation  costs 

3)  Handling,  storage  and  protection  of 
material 

4)  Cost  of  utility  services 

c.  Maintenance  and  Repair  Costs 

TOTAL  VALUE  OF  ALL  RECURRING  COSTS 


Sunk  costs  should  not  be  counted,  but  may  be  shown  separately  for  information. 

Depreciation  should  only  be  used  to  estimate  terminal  values. 

In  general,  a  discount  rate  of  10%  should  be  used  with  a  mid-year  convention.  Certain  specific 
instances  may  call  for  a  different  assumption  but  generally  not  viewed  as  necessary. 


DISCOUNT  RATE  USED  IN  THIS  TABLE 


10.00% 


INT 


Economic  Life 

1 .  The  economic  life  of  a  project  or  asset  is  the  time  during  which  benefits  from  it  may  reasonably 
be  expected  to  accrue  to  the  Air  Force. 

2.  Economic  life  plus  project  lead-time  determine  the  time  period  to  be  considered  when 
conducting  an  economic  analysis. 

3.  Economic  life  of  a  project  or  asset  is  set  by  the  shortest  of  its  physical  life,  technological 
life,  or  mission  life. 

4.  If  necessary,  a  residual  value  will  be  determined  to  be  used  as  a  terminal  value. 

5.  Comparing  assets  with  different  economic  lives. 

a.  Use  terminal  value  for  the  longer  lived  alternative. 

b.  Use  common  denominator  approach  (Least  Common  Multiple  (LCM)  of  lives). 


ECONOMIC  LIFE  USED  IN  THIS  TABLE 

FOR  "NORMAL"  LONG-LIVED  CAPITAL 
ITEMS 

ECONOMIC  LIFE  USED  IN  THIS  TABLE 
FOR  "SHORT"  ECONOMIC  LIVES  SUCH 
SUCH  AS  COMPUTERS 


Sensitivity  analyses  should  be  performed. 
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4.  All  capital  equipment  and  real  property  will  have  a  definable  and  relatively  long 
economic  life. 

5.  The  discount  rate  and  economic  lives  are  defined  by  the  user  and  entered  into  the 
spreadsheets. 


Application:  Benefit-Cost  ratios 

The  table  developed  for  presenting  the  important  elements  of  the  benefit-cost  analysis  is  shown  in 
Table  6.  This  table  establishes  both  the  net  benefit  minus  costs  and  the  benefit-cost  ratio  for  the 
technology  being  evaluated. 

Conclusion 


This  study  developed  a  preliminary  benefits-cost  procedure  and  model  for  the  evaluation  of 
proposed  technology  based  systems  which  are  intended  to  provide  improved  Undergraduate  Pilot 
Training  (UPT)  for  the  USAF.  It  created  the  operational  procedures  necessary  to  acquire  all  data 
required  for  estimating  benefits  and  costs.  In  addition,  in  a  very  preliminary  way,  the  study 
completed  an  operational  test  of  these  procedures  demonstrating  their  feasibility. 

Obvious  areas  requiring  future  work  are  apparent.  These  include  improvements  and  refinements 
in  the  methods  of: 

1 .  acquiring  and  using  the  variables  for  benefits  estimation; 

2.  validating  the  benefits  computation  equations; 

3.  validating  the  application  areas; 

4.  justifying  and  supporting  the  listed  assumptions;  and 

5.  determining  the  general  usability  of  the  operational  procedures. 
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Table  6.  Technology  Based  Enhancements  for  Undergraduate  Pilot  Training 


Name  of  Application  Area  Sample  in  Table  2 

Name  of  Technology  Sample  in  Table  2 

Summary  of  Benefits  Estimation 
Benefits  Estimated  for 


Area  1 

$1,728 

Area  2 

$1,296 

Area  3 

$1,728 

Area  4 

$1,620,000 

Area  5 

$37,843 

Area  6 

$41,472 

TOTAL  ANNUAL  BENEFITS  ESTIMATED  $1 ,704,067|| 

Summary  of  Costs  Estimation 
Costs  Estimated  for: 

Capital  -  Long  Life 
Capital  -  Short  Life 
Annual 


TOTAL  ANNUAL  EQUIVALENT  COSTS  ESTIMATED  $1 .987,141]] 


$4,017,162 

$2,500,000 

$510,000 


NET  ANNUAL  BENEFITS  -  COSTS 

BENEFIT-COST  RATIO  BASED  ON  THE  ABOVE 

Constants  used  in  the  above  estimation 
Discount  Rate 
Long  Life 
Short  Life 


10.00% 

20 

3 


($283,074) 

0.85755]] 
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APPENDIX  A  to  BENEFIT  COST  ANALYSIS 


ESTIMATED  BENEFITS  COMPUTATION  FOR  A  SELECTED  SET  OF 
TECHNOLOGY  APPLICATION  AREAS 


71 


Table  A-1 .  Benefits  Estimation 

Use  of  SIMULATOR  NETWORKING  to  enhance  training  in  OVERHEAD  TRAFFIC  PATTERN  (OTP) 


Estimated  Total  Benefit  of  1  simulator  networking  system  (cockpit)  |  $1,693,829 


Variables,  Constants  and  Intermediate  Computed  Results  Used  for  Benefit  Estimation 

VALUE  RANGE 

NAME  OF  THE  ELEMENT  ASSIGNED  NAME 


Cost  of  the  aircraft  ($/aircraft) 

$8,000,000  AIRC$ 

Marginal  cost  of  flying  the  aircraft  ($/hr) 

$1,500  AIRC$HR 

Probability  of  loss  of  aircraft  (/hr) 

Cost  of  instructor  pilot  time  ($/hr) 

$48||IP$HR 

"Cost"  of  the  IP  ($) 

$10,000,000|IP$ 

Cost  of  trainee  time  ($/hr) 

$20|TRN$HR 

"Cost"  of  the  trainee  ($/trainee) 

$6,000,000|TRNEE$ 

Time  spent  in  OTP  training  (hrs/trainee) 

35||OTP HR 

Transfer  Ratio 

0.3000|TRANRAT 

Operation  days/week/simulator  (dys/wk) 

5||DYS WK 

Operation  hours/day/simulator  (hrs/dy) 

16|HRS DY 

Operation  weeks/year/simulator  (wks/yr) 

50|WKS YR 

Percent  uptime 

0.9000|UP PER 

Percent  time  the  simulator  is  used  for  OTP  training 

1 .0000|OTP PER 

Total  hours  the  simulator  is  used  for  OTP  training  (hrs) 

3600|SIMHRS 

Hours  of  aircraft  time  "lost"  per  year  (hrs/yr) 

j  100|AIRC LST 

Syllabus  hours  for  OTP  training  (hrs/trainee) 

35|SYLBHRS 

Proportion  of  time  the  instructor  flys  in  same  aircraft  as  trainee 

0.8000|jlP PROP 

Proportion  of  time  per  sortie  spent  in  OTP  training 

0.4000||SRT PROP 

Proportion  of  time  the  IP  is  able  to  spend  evaluating  the  trainee  during  flight 

0.6000||EVAL_PR 

Benefits  Estimation  Results  Using  Equation  Estimators 


IDENTIFICATION  OF  EQUATION  AND  EQUATION 

ESTIMATED 

BENEFITS 

RANGE 

NAME 

Benefit  due  to  aircraft  not  being  destroyed 

2592 

$2,592 

AIRC$BEN 

Benefit  due  to  trainee  not  being  killed 

1944 

$1,944 

TRN$BEN 

Benefit  due  to  IP  not  being  killed 

2592 

IP$BEN 

Savings  in  aircraft  use  due  to  simulation 

1620000 

$1,620,000 

AHRS$BEN 

Benefit  due  to  increase  in  OTP  training  time 

25228.8 

$25,229 

TRN$EFF 

Benefit  due  to  increased  efficiency  of  use  of  IP  time 

41472 

$41,472 

IP$EFF 
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Table  A-2.  Training  Systems  Benefits  Estimation  Worksheet 
Use  of  VIDEO  MEDIA  to  enhance  training  in  FORMATION  FLIGHT 


Estimated  Total  Benefit  of  video  media 


$1 30,81 9|| 


Variables,  Constants  and  Intermediate  Computed  Results  Used  for  Benefit  Estimation 

VALUE  RANGE 

NAME  OF  THE  ELEMENT  ASSIGNED  NAME 


Level  of  skill  achieved  under  the  current  syllabus  and  conditions 

0.3000 

L_C 

Number  of  hours  in  the  current  syllabus  for  training  in  this  skill 

25.00 

H_C 

Proportion  of  instructor  hours/student  hours  -  current  syllabus 

0.5000 

P_C 

Cost  of  instructor  pilot  time  ($/hr) 

$48 

L$ 

Cost  of  trainee  time  ($/hr) 

$20 

s_$ 

Level  of  skill  achievable  with  the  proposed  training  system 

0.4000 

L_N 

Number  of  hours  expected  when  using  the  proposed  system 

25.00 

H_N 

Proportion  of  instructor  hours/student  hours  -  proposed  system 

0.5000| 

P_N 

Number  of  students  per  year  undergoing  training 

200| 

N_S 

Number  of  instructor  hours  current  spent  training  this  skill 

2500| 

l_HRS 

Retention  of  skill  under  new  technology 

0.1 8| 

RS_NEW 

Retention  of  skill  under  existing  technology 

0.1 400| 

RS_OLD 

1 

Benefits  Estimation  Results  Using  Equation  Estimators 


ESTIMATED  RANGE 

IDENTIFICATION  OF  EQUATION  AND  EQUATION  BENEFITS  NAME 


Benefit  due  to  training  efficiency  based  on  student  costs 

33333.33333 

STUD_EFF 

Benefit  due  to  training  efficiency  based  on  instructor  costs 

80000 

$80,000 

INS_EFF 

Benefit  due  to  increased  retention  of  skill 

17485.71429 

$17,486 

RSKILL 
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Table  A-3.  Training  Systems  Benefits  Estimation  Worksheet 

Use  of  ELECTRONIC  KNEEPAD  to  enhance  training  in  FORMATION  FLIGHT 


Estimated  Total  Benefit  of  electronic  kneepad 


$107,8461 


Variables,  Constants  and  Intermediate  Computed  Results  Used  for  Benefit  Estimation 
NAME  OF  THE  ELEMENT _ =„__=_  1 


Level  of  skill  achieved  under  the  current  syllabus  and  conditions _ 


Number  of  hours  in  the  current  syllabus  for  training  in  this  skill _ 


Proportion  of  instructor  hours/student  hours  -  current  syllabus 


Cost  of  instructor  pilot  time  ($/hr) _ _ 


Cost  of  trainee  time  ($/hr)  _ 


Level  of  skill  achievable  with  the  proposed  training  system _ 


Number  of  hours  expected  when  using  the  proposed  system _ __ 


Proportion  of  instructor  hours/student  hours  -  proposed  system 


Number  of  students  per  year  undergoing  training  _ 


Number  of  instructor  hours  currently  spent  training  this  skill _ _ 


Retention  of  skill  under  new  technology _ 


Retention  of  skill  under  existing  technology  


VALUE  RANGE 
ASSIGNED  NAME 


Benefits  Estimation  Results  Using  Equation  Estimators 


ESTIMATED  RANGE 


IL/U.M  I  •»  —  . . . .  . . . . — - 

Benefit  due  to  training  efficiency  based  on  student  costs 

26666.66667 

$26,667 

STUD_EFF 

Benefit  due  to  training  efficiency  based  on  instructor  costs 

64000 

$64,000 

INS_EFF 

Benefit  due  to  increased  retention  of  skill 

17178.94737 

$17,179 

RSKILL 

Table  A-4.  Training  Systems  Benefits  Estimation  Worksheet 

Use  of  AERONAUTICAL  TRAINING  RECORDER  to  enhance  training  in  FORMATION  FLIGHT 


Estimated  Total  Benefit  of  the  Aeronautical  Training  Recorder  ||  $1 1 7,980 

Variables,  Constants  and  Intermediate  Computed  Results  Used  for  Benefit  Estimation 


VALUE  RANGE 

NAME  OF  THE  ELEMENT  ASSIGNED  NAME 

Level  of  skill  achieved  under  the  current  syllabus  and  conditions 

0.3000||L C 

Number  of  hours  in  the  current  syllabus  for  training  in  this  skill 

25.00||H C 

Proportion  of  instructor  hours/student  hours  -  current  syllabus 

0.5000|P C 

Cost  of  instructor  pilot  time  ($/hr) 

$48|l$ 

Cost  of  trainee  time  ($/hr) 

$20||S$ 

Level  of  skill  achievable  with  the  proposed  training  system 

0.3500|L N 

Number  of  hours  expected  when  using  the  proposed  system 

23.00|H N 

Proportion  of  instructor  hours/student  hours  -  proposed  system 

0.5000||P N 

Number  of  students  per  year  undergoing  training 

200|N S 

Number  of  instructor  hours  currently  spent  training  this  skill 

0|jl  HRS 

Retention  of  skill  under  new  technology 

0.60|RS NEW 

Retention  of  skill  under  existing  technology 

0.26|RS OLD 

Benefits  Estimation  Results  Using  Equation  Estimators 

ESTIMATED  RANGE 

IDENTIFICATION  OF  EQUATION  AND  EQUATION  BENEFITS  NAME 

Benefit  due  to  training  efficiency  based  on  student  costs 

24666.66667 

Benefit  due  to  training  efficiency  based  on  instructor  costs 

59200 

$59,20o|  INS EFF 

Benefit  due  to  increased  retention  of  skill 

34113.71237 

$34,114||rSKILL 
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Table  A-5.  Benefits  Estimation 

Use  of  SIMULATOR  NETWORKING  to  enhance  training  in  FORMATION  FLYING 
Estimated  Total  Benefit  of  1  simulator  networking  system  (cockpit)  ||  $1,704,067| 


Variables,  Constants  and  Intermediate  Computed  Results  Used  for  Benefit  Estimation 


VALUE  RANGE 

NAME  OF  THE  ELEMENT  ASSIGNED  NAME 

Cost  of  the  aircraft  ($/aircraft) 

$8,000,000 

AIRC$ 

Marginal  cost  of  flying  the  aircraft  ($/hr) 

$1,500 

AIRC$HR 

Probability  of  loss  of  aircraft  (/hr) 

0.0000002 

AIRCPROB 

Cost  of  instructor  pilot  time  ($/hr) 

$48 

IP$HR 

"Cost"  of  the  IP  ($) 

$10,000,000 

IP$  I 

Cost  of  trainee  time  ($/hr) 

$20 

TRN$_HR 

"Cost"  of  the  trainee  ($/trainee) 

$6,000,000 

TRNEE$ 

Time  spent  in  formation  training  (hrs/trainee) 

40 

FORM_HR 

Transfer  Ratio 

0.3000 

TRANRAT 

Operation  days/week/simulator  (dys/wk) 

5 

DYS WK 

Operation  hours/day/simulator  (hrs/dy) 

16 

HRS DY 

Operation  weeks/year/simulator  (wks/yr) 

50 

WKS YR 

Percent  uptime 

0.9000 

UP_PER 

Percent  time  the  simulator  is  used  for  formation  flying  training 

1 .0000 

FORM_PER 

Total  hours  the  simulator  is  used  for  formation  training  (hrs) 

3600 

SIMHRS 

Hours  of  aircraft  time  "lost"  per  year  (hrs/yr) 

100 

AIRC_LST 

Syllabus  hours  for  formation  training  (hrs/trainee) 

40 

SYLBHRS  | 

Proportion  of  time  the  instructor  flys  in  same  aircraft  as  trainee 

0.8000 

Proportion  of  time  per  sortie  spent  in  formation  flying  training 

0.6000 

SRT PROP 

Proportion  of  time  the  IP  is  able  to  spend  evaluating  the 
trainee  during  flight 

0.6000 

EVAL_PR 

Benefits  Estimation  Results  Using  Equation  Estimators 

IDENTIFICATION  OF  EQUATION  AND  EQUATION 

ESTIMATED  RANGE 

BENEFITS  NAME 

Benefit  due  to  aircraft  not  being  destroyed 

1728 

$1,728 

AIRC$BEN 

Benefit  due  to  trainee  not  being  killed 

1296 

$1,296 

TRN$BEN 

Benefit  due  to  IP  not  being  killed 

1728 

$1,728 

IP$BEN 

Savings  in  aircraft  use  due  to  simulation 

1620000 

$1,620,000 

AHRS$BEN 

Benefit  due  to  increase  in  formation  training  time 

37843.2 

TRN$EFF 

Benefit  due  to  increased  efficiency  of  use  of  IP  time 

41472 

IP$_EFF 
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Table  A-6.  Benefits  Estimation 

Use  of  SIMULATOR  NETWORKING  to  enhance  training  in  LOW-ALTITUDE  FLYING  (LAF) 


Estimated  Total  Benefit  of  1  simulator  networking  system  (cockpit) 


$1,683,979 


Variables,  Constants  and  Intermediate  Computed  Results  Used  for  Benefit  Estimation 


NAME  OF  THE  ELEMENT 


VALUE 

ASSIGNED 


RANGE 

NAME 


Cost  of  the  aircraft  ($/aircraft) 

OO 

o 

o 

o 

o 

o 

o 

AIRC$ 

Marginal  cost  of  flying  the  aircraft  ($/hr) 

$1,500 

AIRC$_HR 

Probability  of  loss  of  aircraft  (/hr) 

0.0000002 

AIRCPROB 

Cost  of  instructor  pilot  time  ($/hr) 

$48 

QlSSliH 

"Cost"  of  the  IP($) 

$10,000,000 

IP$ 

Cost  of  trainee  time  ($/hr) 

$20 

TRN$HR 

"Cost”  of  the  trainee  ($/trainee) 

$6,000,000 

TRNEE$ 

Time  spent  in  LAF  training  (hrs/trainee) 

30 

LAF_HR 

Transfer  Ratio 

0.3000 

TRANRAT 

Operation  days/week/simulator  (dys/wk) 

5| 

DYS_WK 

Operation  hours/day/simulator  (hrs/dy) 

16 

Operation  weeks/year/simulator  (wks/yr) 

50 

WKS_YR 

Percent  uptime 

0.9000| 

UP_PER 

Percent  time  the  simulator  is  used  for  LAF  training 

1  .ooooj 

LAF_PER 

Total  hours  the  simulator  is  used  for  LAF  training  (hrs) 

360 oj 

SIMHRS 

Hours  of  aircraft  time  "lost"  per  year  (hrs/yr) 

looj 

AIRC_LST 

Syllabus  hours  for  LAF  training  (hrs/trainee) 

37| 

SYLBHRS 

Proportion  of  time  the  instructor  flys  in  same  aircraft  as  trainee 

0.7000| 

IP_PROP 

Proportion  of  time  per  sortie  spent  in  LAF  training 

0.4000| 

SRT_PROP 

Proportion  of  time  the  IP  is  able  to  spend  evaluating  the 
trainee  during  flight 

0.600o| 

EVAL PR 

Benefits  Estimation  Results  Using  Equation  Estimators 


ESTIMATED  RANGE 

IDENTIFICATION  OF  EQUATION  AND  EQUATION  BENEFITS  NAME 


Benefit  due  to  aircraft  not  being  destroyed 

1728 

$1,728 

AIRC$BEN 

Benefit  due  to  trainee  not  being  killed 

1296 

$1 ,296 

TRN$BEN 

Benefit  due  to  IP  not  being  killed 

1512 

$1,512 

IP$BEN 

Savings  in  aircraft  use  due  to  simulation 

1620000 

$1,620,000 

AHRS$BEN 

Benefit  due  to  increase  in  LAF  training  time 

23155.2 

$23,155 

TRN$EFF 

Benefit  due  to  increased  efficiency  of  use  of  IP  time 

36288 

$36,288 

IP$_EFF 
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Table  A-7.  Benefits  Estimation 

Use  of  the  SIMULATOR  NETWORKING  SYSTEM  to  enhance  continuation  training  for  the  INSTRUCTOR 
PILOT  (IP) 

Estimated  Total  Benefit  of  1  simulator  networking  system  (cockpit)  $1,132,070| 

Variables,  Constants  and  Intermediate  Computed  Results  Used  for  Benefit  Estimation 

VALUE  RANGE 


NAME  OF  THE  ELEMENT 

ASSIGNED 

NAME 

Cost  of  the  aircraft  ($/aircraft) 

$8,000,000 

AIRC$ 

Marginal  cost  of  flying  the  aircraft  ($/hr) 

$1,500 

AIRC$HR 

Probability  of  loss  of  aircraft  (/hr) 

AIRCPROB 

Cost  of  instructor  pilot  time  ($/hr) 

$48 

IP$_HR 

"Cost"  of  the  IP  ($) 

$10,000,000 

IP$ 

Cost  of  trainee  time  ($/hr) 

$q 

TRN$_HR 

"Cost"  of  the  trainee  ($/trainee) 

$o| 

TRNEE$ 

Time  spent  in  IP  training  (hrs/IP) 

IP_HR 

Transfer  Ratio 

TRANRAT 

Operation  days/week/simulator  (dys/wk) 

5 

DYS_WK 

Operation  hours/day/simulator  (hrs/dy) 

16 

HRS DY 

Operation  weeks/year/simulator  (wks/yr) 

50 

WKS YR 

Percent  uptime 

0.9000 

UP.PER 

Percent  time  the  simulator  is  used  for  IP  training 

1.0000 

IP_PER 

Total  hours  the  simulator  is  used  for  IP  training  (hrs) 

SIMHRS 

Hours  of  aircraft  time  "lost"  per  year  (hrs/yr) 

100| 

AIRC_LST 

Syllabus  hours  for  IP  training  (hrs/IP) 

20 

SYLBHRS 

Proportion  of  time  the  instructor  fiys  the  aircraft 

0.8000 

IP PROP 

Proportion  of  time  per  sortie  spent  in  IP  training 

0.8000 

SRT_PROP 

Proportion  of  time  the  IP  is  able  to  spend  evaluating  the 
trainee  during  flight 

0.0000 

EVAL_PR 

Benefits  Estimation  Results  Using  Equation  Estimators 


IDENTIFICATION  OF  EQUATION  AND  EQUATION 

ESTIMATED 

BENEFITS 

RANGE 

NAME 

Benefit  due  to  aircraft  not  being  destroyed 

1152 

$1,152 

AIRC$BEN 

Benefit  due  to  trainee  not  being  killed 

0 

$0 

TRN$BEN 

Benefit  due  to  IP  not  being  killed 

1152 

$1,152 

IP$BEN 

Savings  in  aircraft  use  due  to  simulation 

1080000 

$1 ,080,000 

AHRS$BEN 

Benefit  due  to  increase  in  IP  training  time 

22118.4 

$22,118 

TRN$_EFF 

Benefit  due  to  increased  efficiency  of  use  of  IP  time 

27648 

$27,648 

IP$EFF 
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